Design of Experiments (Active Learning) with Bayesian Optimization
Contents
Design of Experiments (Active Learning) with Bayesian Optimization#
Design of Experiments (DoE), or active learning, is the research area of choosing which parameters to sample to learn about or optimize some system (usually an experiment or expensive simulation).
A few common use cases:
We want to maximize/minimize some metric
Tune elemental composition to optimize some property
Fine the best temperature to operate some process
Build a surrogate (fast) model for some experimental process
Develop a ML model that can reproduce the output of a large chemical plant
We will repeatedly sample a process and use that information to select new points to sample. We will try to accomplish some goal with the minimum number of evaluations. There are many strategies that we can use.
Trial Function#
As a trial function, let’s try to find the highest value of the function
where \(\epsilon\) is random noise with a scale of 0.1. We’ll limit ourselves to the region \(x\in[0,1]\).
Let’s define and plot the objective function first!
import numpy as np
import plotly.express as px
def objective(x, noise=0.05):
noise = np.random.normal(loc=0, scale=noise, size=x.shape)
return (x**2 * np.sin(5 * np.pi * x) ** 6.0) + noise
# Plot the objective function evaluated at 100 evenly spaced points
x_eval = np.linspace(0, 1, 100)
fig = px.scatter(x=x_eval, y=objective(x_eval))
# Add a line for the actual function evaluation!
x_eval = np.linspace(0, 1, 1000)
fig.add_scatter(x=x_eval, y=objective(x_eval, noise=0))
There are five local maxima in this range. The noise is significant here. The best maxima is the one to the right. We got this by doing 100 function evaluations.
Uniform or random sampling strategies#
If we don’t want to build a model or do local optimizations, we can try to optimize a function \(f(x,y)\) by randomly sampling with various parameters. Three of the most common strategies are:
random sampling: Just pick random parameters and try them
latin hypercube sampling: try to pick points to improve the variability of the outputs
Sobol sampling: try to pick random points with the goal of equally covering the entire space and then going back and filling in the space
These are pretty simple, but usually helpful for selecting the first points to try. A rough rule of thumb is to sample ~5-10 or ~2D (where D is the number of dimensions) points before trying to use a fancy ML based method if you don’t know anything about the system. Let’s visualize the strategies in two dimensions!
import numpy as np
from scipy.stats import qmc
N = 50
# Generate N random samples in two dimensions
random_samples = np.random.uniform(0, 1, size=(N, 2))
# Generate N latin hypercube samples
latin_hypercube_samples = qmc.LatinHypercube(2).random(N)
# Generate N sobol samples in two dimensions
sobol_samples = qmc.Sobol(d=2, scramble=True).random(N)
/tmp/ipykernel_1160/2070957824.py:13: UserWarning:
The balance properties of Sobol' points require n to be a power of 2.
Now let’s make a little animation to see how the differenget
# Make a matplotlib animation!
import matplotlib.pyplot as plt
import numpy as np
from matplotlib import animation, rc
# Make the plots
fig, ax = plt.subplots(ncols=3, nrows=1, figsize=(16, 9))
(pt_random,) = ax[0].plot([], [], "ro")
(pt_latin_hypercube,) = ax[1].plot([], [], "bo")
(pt_sobol,) = ax[2].plot([], [], "go")
ax[0].set_xlim(0, 1)
ax[0].set_ylim(0, 1)
ax[0].set_adjustable("box")
ax[0].set_aspect("equal")
ax[0].set_title("Random")
ax[1].set_xlim(0, 1)
ax[1].set_ylim(0, 1)
ax[1].set_aspect("equal")
ax[1].set_adjustable("box")
ax[1].set_title("Latin Hypercube")
ax[2].set_xlim(0, 1)
ax[2].set_ylim(0, 1)
ax[2].set_aspect("equal")
ax[2].set_adjustable("box")
ax[2].set_title("Sobol")
# Initialize and clear the data!
def init():
pt_random.set_data([], [])
pt_latin_hypercube.set_data([], [])
pt_sobol.set_data([], [])
return (pt_random, pt_latin_hypercube, pt_sobol)
def animate(i):
# Set the data for a given frame
pt_random.set_data(random_samples[:i, 0], random_samples[:i, 1])
pt_latin_hypercube.set_data(
latin_hypercube_samples[:i, 0], latin_hypercube_samples[:i, 1]
)
pt_sobol.set_data(sobol_samples[:i, 0], sobol_samples[:i, 1])
return (pt_random, pt_latin_hypercube, pt_sobol)
# Make the animation!
anim = animation.FuncAnimation(
fig,
animate,
init_func=init,
frames=N,
interval=1000,
repeat_delay=5000,
blit=True,
)
rc("animation", html="jshtml")
anim
Evaluation of simple sampling strategies on optimizing the test function#
Let’s see how these three strategies work for sampling the test function above.
Base case: random sampling#
The easiest thing we can do is randomly pick points in the range. For a given number of random samples, we can evaluate all of them and see what the best is!
import pandas as pd
# Make an empty dataframe!
df_random = pd.DataFrame()
num_samples = list(range(1, 100))
for n in num_samples:
# Run 10 trials with n samples!
y = [np.max(objective(np.random.uniform(0, 1, size=(n,)))) for i in range(100)]
# Add the results to the dataframe
df_random = pd.concat(
(
df_random,
pd.DataFrame(
{"num_samples": [n], "objective": [np.mean(y)], "stdev": [np.std(y)]}
),
),
ignore_index=True,
)
# Plot with error bars!
px.line(df_random, x="num_samples", y="objective", error_y="stdev")
Note that our process - pick the best value of the random points is actually not giving us quite the right answer when we evaluate many times, since we also have a bit of noise in the measurements!
We can see that to get close to the real value of ~0.80 we need ~80 function evaluations. At 20 evaluations our best value is ~0.65.
Latin hypercube sampling#
import pandas as pd
import plotly.graph_objects as go
from scipy.stats import qmc
# Make an empty dataframe!
df_latinhypercube = pd.DataFrame()
num_samples = list(range(1, 100))
for n in num_samples:
# Run 10 trials with n samples!
y = [np.max(objective(qmc.LatinHypercube(2).random(n))) for i in range(100)]
# Add the results to the dataframe
df_latinhypercube = pd.concat(
(
df_latinhypercube,
pd.DataFrame(
{"num_samples": [n], "objective": [np.mean(y)], "stdev": [np.std(y)]}
),
),
ignore_index=True,
)
fig = go.Figure()
fig.add_scatter(
x=df_random["num_samples"],
y=df_random["objective"],
error_y={"array": df_random["stdev"]},
name="Random sampling",
)
fig.add_scatter(
x=df_latinhypercube["num_samples"],
y=df_latinhypercube["objective"],
error_y={"array": df_latinhypercube["stdev"]},
name="Latin Hypercube Sampling",
)
Notice how much better latin hypercube sampling works for this process! At 20 evaluations our best value is ~0.73. The gap is pretty consistent at all sample sizes.
Sobol Sampling#
import pandas as pd
import plotly.graph_objects as go
from scipy.stats import qmc
# Make an empty dataframe!
df_sobol = pd.DataFrame()
num_samples = list(range(1, 100))
for n in num_samples:
# Run 10 trials with n samples!
y = [np.max(objective(qmc.Sobol(d=1, scramble=True).random(n))) for i in range(100)]
# Add the results to the dataframe
df_sobol = pd.concat(
(
df_sobol,
pd.DataFrame(
{"num_samples": [n], "objective": [np.mean(y)], "stdev": [np.std(y)]}
),
),
ignore_index=True,
)
fig = go.Figure()
fig.add_scatter(
x=df_random["num_samples"],
y=df_random["objective"],
error_y={"array": df_random["stdev"]},
name="Random sampling",
)
fig.add_scatter(
x=df_latinhypercube["num_samples"],
y=df_latinhypercube["objective"],
error_y={"array": df_latinhypercube["stdev"]},
name="Latin Hypercube Sampling",
)
fig.add_scatter(
x=df_sobol["num_samples"],
y=df_sobol["objective"],
error_y={"array": df_sobol["stdev"]},
name="Sobol sampling",
)
/tmp/ipykernel_1160/1287512983.py:13: UserWarning:
The balance properties of Sobol' points require n to be a power of 2.
Bayesian Optimization with scikit-optimize#
Bayesian optimization is one approach in a field of methods called “sequential model based optimization”. The general idea is
Generate a few (~2-3 times the number of variables?) random configurations in your range of interest and evaluate them
Fit the best Gaussian Process you can to your data
Ask the GP which new value is most likely to improve on the current best known point (expected improvement)
Sample that point and add it to the dataset
Go back to 2.
This is one of many, many, many possible strategies you can use! The key is to trade-off exploration vs exploitation
exploration: try something new you know little about and hope it is significantly better than your current best
exploitation: try to incrementally improve your current best solution
scikit-optimize is one framework among many for handling this loop using various types of Gaussian Process models.
See also
https://scikit-optimize.github.io/stable/index.html
from skopt import gp_minimize
def objective(x, noise=0.05):
(x,) = x
noise = np.random.normal(loc=0, scale=noise)
# Notice the - sign! we want to maximize this
return -(x**2 * np.sin(5 * np.pi * x) ** 6.0) + noise
res = gp_minimize(
objective, # the function to minimize
[(0.0, 1.0)], # the bounds on each dimension of x
acq_func="EI", # the acquisition function
n_calls=30, # the number of evaluations of f
n_random_starts=5, # the number of random initialization points
noise=0.1**2, # the noise level (optional)
initial_point_generator="lhs"
) # the random seed
from skopt.plots import plot_convergence
plot_convergence(res);
from skopt.plots import plot_gaussian_process
for n_iter in range(10):
# Plot true function.
plt.subplot(10, 2, 2 * n_iter + 1)
if n_iter == 0:
show_legend = True
else:
show_legend = False
ax = plot_gaussian_process(
res,
n_calls=n_iter,
objective=lambda x: objective(x, 0),
noise_level=0.05,
show_legend=show_legend,
show_title=False,
show_next_point=False,
show_acq_func=False,
)
ax.set_ylabel("")
ax.set_xlabel("")
# Plot EI(x)
plt.subplot(10, 2, 2 * n_iter + 2)
ax = plot_gaussian_process(
res,
n_calls=n_iter,
show_legend=show_legend,
show_title=False,
show_mu=False,
show_acq_func=True,
show_observations=False,
show_next_point=True,
)
ax.set_ylabel("")
ax.set_xlabel("")
fig = plt.gcf()
fig.set_figheight(30)
fig.set_figwidth(10)
plt.show()
Let’s compare this to our much simpler sampling strategies!
fig = go.Figure()
fig.add_scatter(
x=df_random["num_samples"],
y=df_random["objective"],
# error_y={"array": df_random["stdev"]},
name="Random sampling",
)
fig.add_scatter(
x=df_latinhypercube["num_samples"],
y=df_latinhypercube["objective"],
# error_y={"array": df_latinhypercube["stdev"]},
name="Latin Hypercube Sampling",
)
fig.add_scatter(
x=df_sobol["num_samples"],
y=df_sobol["objective"],
# error_y={"array": df_sobol["stdev"]},
name="Sobol sampling",
)
fig.add_scatter(
x=list(range(len(res.func_vals))),
y=np.maximum.accumulate(-res.func_vals),
name="scikit-optimize",
)
from skopt.optimizer import Optimizer
optimizer = Optimizer(5, base_estimator,
n_initial_points=5,
initial_point_generator="sobol",
n_jobs=n_jobs,
acq_func=acq_func, acq_optimizer=acq_optimizer,
random_state=random_state,
model_queue_size=model_queue_size,
acq_optimizer_kwargs=acq_optimizer_kwargs,
acq_func_kwargs=acq_func_kwargs)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Cell In [10], line 3
1 from skopt.optimizer import Optimizer
----> 3 optimizer = Optimizer(5, base_estimator,
4 n_initial_points=5,
5 initial_point_generator="sobol",
6 n_jobs=n_jobs,
7 acq_func=acq_func, acq_optimizer=acq_optimizer,
8 random_state=random_state,
9 model_queue_size=model_queue_size,
10 acq_optimizer_kwargs=acq_optimizer_kwargs,
11 acq_func_kwargs=acq_func_kwargs)
NameError: name 'base_estimator' is not defined
res.ask()
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
File /opt/conda/lib/python3.9/site-packages/scipy/optimize/_optimize.py:124, in OptimizeResult.__getattr__(self, name)
123 try:
--> 124 return self[name]
125 except KeyError as e:
KeyError: 'ask'
The above exception was the direct cause of the following exception:
AttributeError Traceback (most recent call last)
Cell In [92], line 1
----> 1 res.ask()
File /opt/conda/lib/python3.9/site-packages/scipy/optimize/_optimize.py:126, in OptimizeResult.__getattr__(self, name)
124 return self[name]
125 except KeyError as e:
--> 126 raise AttributeError(name) from e
AttributeError: ask
The first five points are basically indistinguishable from the other strategies, but then the optimization makes this much more efficient.
Bayesian Optimization with optuna#
Bayesian optimization is one approach in a field of methods called “sequential model based optimization”. The general idea is
Generate a few (~2-3 times the number of variables?) random configurations in your range of interest and evaluate them
Fit the best Gaussian Process you can to your data
Ask the GP which new value is most likely to improve on the current best known point (expected improvement)
Sample that point and add it to the dataset
Go back to 2.
This is one of many, many, many possible strategies you can use! The key is to trade-off exploration vs exploitation
exploration: try something new you know little about and hope it is significantly better than your current best
exploitation: try to incrementally improve your current best solution
optuna is one framework among many for handling this loop using various types of Gaussian Process models.
See also
https://optuna.readthedocs.io/en/stable/tutorial/index.html
import optuna
from optuna.visualization import plot_optimization_history, plot_slice
def objective(trial):
x = trial.suggest_float("x", 0, 1)
noise = np.random.normal(loc=0, scale=0.05)
return (x**2 * np.sin(5 * np.pi * x) ** 6.0) + noise
study = optuna.create_study(direction="maximize")
study.optimize(objective, n_trials=50)
print(study.best_params)
fig = plot_optimization_history(study)
fig.show()
fig = plot_slice(study)
fig.show()
[I 2022-10-07 20:10:03,864] A new study created in memory with name: no-name-ecea0f5c-0d81-4c9c-aa1d-ae984c99a303
[I 2022-10-07 20:10:03,867] Trial 0 finished with value: -0.008840609506676953 and parameters: {'x': 0.44725537554532047}. Best is trial 0 with value: -0.008840609506676953.
[I 2022-10-07 20:10:03,869] Trial 1 finished with value: 0.03536105539186854 and parameters: {'x': 0.7585213066071057}. Best is trial 1 with value: 0.03536105539186854.
[I 2022-10-07 20:10:03,871] Trial 2 finished with value: -0.007745524638831758 and parameters: {'x': 0.17262379907732517}. Best is trial 1 with value: 0.03536105539186854.
[I 2022-10-07 20:10:03,873] Trial 3 finished with value: 0.19745840855203953 and parameters: {'x': 0.5004863546523648}. Best is trial 3 with value: 0.19745840855203953.
[I 2022-10-07 20:10:03,875] Trial 4 finished with value: 0.04617566855268616 and parameters: {'x': 0.1552959525788351}. Best is trial 3 with value: 0.19745840855203953.
[I 2022-10-07 20:10:03,877] Trial 5 finished with value: 0.02632757742973362 and parameters: {'x': 0.776334726942192}. Best is trial 3 with value: 0.19745840855203953.
[I 2022-10-07 20:10:03,879] Trial 6 finished with value: 0.006467007970310293 and parameters: {'x': 0.24933492667818846}. Best is trial 3 with value: 0.19745840855203953.
[I 2022-10-07 20:10:03,880] Trial 7 finished with value: 0.08906411977963867 and parameters: {'x': 0.9492989182647352}. Best is trial 3 with value: 0.19745840855203953.
[I 2022-10-07 20:10:03,885] Trial 8 finished with value: -0.09798342373788475 and parameters: {'x': 0.205179394876144}. Best is trial 3 with value: 0.19745840855203953.
[I 2022-10-07 20:10:03,887] Trial 9 finished with value: 0.09972672536096322 and parameters: {'x': 0.30045315683689944}. Best is trial 3 with value: 0.19745840855203953.
[I 2022-10-07 20:10:03,892] Trial 10 finished with value: 0.039248047495849404 and parameters: {'x': 0.5650679454664995}. Best is trial 3 with value: 0.19745840855203953.
[I 2022-10-07 20:10:03,898] Trial 11 finished with value: -0.021810170499334154 and parameters: {'x': 0.4420078832292691}. Best is trial 3 with value: 0.19745840855203953.
[I 2022-10-07 20:10:03,903] Trial 12 finished with value: -0.06971581862332217 and parameters: {'x': 0.02445352213363744}. Best is trial 3 with value: 0.19745840855203953.
[I 2022-10-07 20:10:03,908] Trial 13 finished with value: -0.009623931568281138 and parameters: {'x': 0.590236957112998}. Best is trial 3 with value: 0.19745840855203953.
[I 2022-10-07 20:10:03,913] Trial 14 finished with value: 0.029236096743635278 and parameters: {'x': 0.35925086292196295}. Best is trial 3 with value: 0.19745840855203953.
[I 2022-10-07 20:10:03,918] Trial 15 finished with value: 0.028363508756375857 and parameters: {'x': 0.32839167758872656}. Best is trial 3 with value: 0.19745840855203953.
[I 2022-10-07 20:10:03,923] Trial 16 finished with value: 0.4920504049950435 and parameters: {'x': 0.7017885259655552}. Best is trial 16 with value: 0.4920504049950435.
[I 2022-10-07 20:10:03,927] Trial 17 finished with value: 0.4735784894459026 and parameters: {'x': 0.6967632642011069}. Best is trial 16 with value: 0.4920504049950435.
[I 2022-10-07 20:10:03,932] Trial 18 finished with value: 0.37489238543512465 and parameters: {'x': 0.7204685062008253}. Best is trial 16 with value: 0.4920504049950435.
[I 2022-10-07 20:10:03,937] Trial 19 finished with value: 0.07653996412051933 and parameters: {'x': 0.9551265488312775}. Best is trial 16 with value: 0.4920504049950435.
[I 2022-10-07 20:10:03,942] Trial 20 finished with value: 0.23450513208237592 and parameters: {'x': 0.6746452887659607}. Best is trial 16 with value: 0.4920504049950435.
[I 2022-10-07 20:10:03,946] Trial 21 finished with value: 0.023798782615140738 and parameters: {'x': 0.8178345490604649}. Best is trial 16 with value: 0.4920504049950435.
[I 2022-10-07 20:10:03,950] Trial 22 finished with value: 0.3246676205932298 and parameters: {'x': 0.6781525078826856}. Best is trial 16 with value: 0.4920504049950435.
[I 2022-10-07 20:10:03,954] Trial 23 finished with value: 0.2923104904346908 and parameters: {'x': 0.867501035488938}. Best is trial 16 with value: 0.4920504049950435.
[I 2022-10-07 20:10:03,958] Trial 24 finished with value: 0.48575340833070724 and parameters: {'x': 0.6908111306606358}. Best is trial 16 with value: 0.4920504049950435.
[I 2022-10-07 20:10:03,963] Trial 25 finished with value: -0.021436327289518258 and parameters: {'x': 0.6061196784646283}. Best is trial 16 with value: 0.4920504049950435.
[I 2022-10-07 20:10:03,967] Trial 26 finished with value: 0.9272993784066883 and parameters: {'x': 0.9034052301694635}. Best is trial 26 with value: 0.9272993784066883.
[I 2022-10-07 20:10:03,971] Trial 27 finished with value: 0.43774938319159185 and parameters: {'x': 0.8723863017087699}. Best is trial 26 with value: 0.9272993784066883.
[I 2022-10-07 20:10:03,975] Trial 28 finished with value: 0.5845741559476502 and parameters: {'x': 0.8784438920594411}. Best is trial 26 with value: 0.9272993784066883.
[I 2022-10-07 20:10:03,979] Trial 29 finished with value: 0.757833109941485 and parameters: {'x': 0.8887022145012182}. Best is trial 26 with value: 0.9272993784066883.
[I 2022-10-07 20:10:03,983] Trial 30 finished with value: -0.043489657494935066 and parameters: {'x': 0.9866182407087292}. Best is trial 26 with value: 0.9272993784066883.
[I 2022-10-07 20:10:03,986] Trial 31 finished with value: 0.5425407005557477 and parameters: {'x': 0.8818321093221066}. Best is trial 26 with value: 0.9272993784066883.
[I 2022-10-07 20:10:03,990] Trial 32 finished with value: 0.5008648224159903 and parameters: {'x': 0.874429724052731}. Best is trial 26 with value: 0.9272993784066883.
[I 2022-10-07 20:10:03,993] Trial 33 finished with value: 0.7786602841446392 and parameters: {'x': 0.9088656534692047}. Best is trial 26 with value: 0.9272993784066883.
[I 2022-10-07 20:10:03,997] Trial 34 finished with value: -0.02988018567529197 and parameters: {'x': 0.8062056808707349}. Best is trial 26 with value: 0.9272993784066883.
[I 2022-10-07 20:10:04,000] Trial 35 finished with value: 0.3338275756503301 and parameters: {'x': 0.9321346826612424}. Best is trial 26 with value: 0.9272993784066883.
[I 2022-10-07 20:10:04,003] Trial 36 finished with value: 0.05549465034892793 and parameters: {'x': 0.9901606619226757}. Best is trial 26 with value: 0.9272993784066883.
[I 2022-10-07 20:10:04,007] Trial 37 finished with value: -0.006075907144603994 and parameters: {'x': 0.8031054144674614}. Best is trial 26 with value: 0.9272993784066883.
[I 2022-10-07 20:10:04,012] Trial 38 finished with value: 0.7890227607663236 and parameters: {'x': 0.9092460569275483}. Best is trial 26 with value: 0.9272993784066883.
[I 2022-10-07 20:10:04,017] Trial 39 finished with value: 0.059187479463921784 and parameters: {'x': 0.7629144764426539}. Best is trial 26 with value: 0.9272993784066883.
[I 2022-10-07 20:10:04,023] Trial 40 finished with value: 0.4650605361094253 and parameters: {'x': 0.9255855490245402}. Best is trial 26 with value: 0.9272993784066883.
[I 2022-10-07 20:10:04,027] Trial 41 finished with value: 0.0608062769213477 and parameters: {'x': 0.8417063235045692}. Best is trial 26 with value: 0.9272993784066883.
[I 2022-10-07 20:10:04,030] Trial 42 finished with value: 0.6725516613641646 and parameters: {'x': 0.917316864233827}. Best is trial 26 with value: 0.9272993784066883.
[I 2022-10-07 20:10:04,034] Trial 43 finished with value: 0.0809357729117755 and parameters: {'x': 0.9484827626118767}. Best is trial 26 with value: 0.9272993784066883.
[I 2022-10-07 20:10:04,037] Trial 44 finished with value: 0.8043466005458887 and parameters: {'x': 0.9080197171600985}. Best is trial 26 with value: 0.9272993784066883.
[I 2022-10-07 20:10:04,040] Trial 45 finished with value: 0.08687235315604201 and parameters: {'x': 0.7713502144854819}. Best is trial 26 with value: 0.9272993784066883.
[I 2022-10-07 20:10:04,044] Trial 46 finished with value: -0.0244058682779752 and parameters: {'x': 0.9791087310978873}. Best is trial 26 with value: 0.9272993784066883.
[I 2022-10-07 20:10:04,048] Trial 47 finished with value: 0.7944389674170577 and parameters: {'x': 0.9101116021921416}. Best is trial 26 with value: 0.9272993784066883.
[I 2022-10-07 20:10:04,051] Trial 48 finished with value: 0.15364889665838272 and parameters: {'x': 0.47795541686405907}. Best is trial 26 with value: 0.9272993784066883.
[I 2022-10-07 20:10:04,054] Trial 49 finished with value: 0.02369391410298022 and parameters: {'x': 0.8361289086688972}. Best is trial 26 with value: 0.9272993784066883.
{'x': 0.9034052301694635}
# Plot with error bars!
y_trials_stacked = np.maximum.accumulate(np.array(y_trials).T)
import plotly.graph_objects as go
fig = plot_optimization_history(study)
fig.add_scatter(
x=df_random["num_samples"],
y=df_random["objective"],
error_y={"array": df_random["stdev"]},
name="Random sampling",
)
fig.add_scatter(
x=df_latinhypercube["num_samples"],
y=df_latinhypercube["objective"],
error_y={"array": df_latinhypercube["stdev"]},
name="Latin Hypercube Sampling",
)
fig.add_scatter(
x=df_sobol["num_samples"],
y=df_sobol["objective"],
error_y={"array": df_sobol["stdev"]},
name="Sobol sampling",
)
Bayesian Optimization with ax#
Bayesian optimization is one approach in a field of methods called “sequential model based optimization”. The general idea is
Generate a few (~2-3 times the number of variables?) random configurations in your range of interest and evaluate them
Fit the best Gaussian Process you can to your data
Ask the GP which new value is most likely to improve on the current best known point (expected improvement)
Sample that point and add it to the dataset
Go back to 2.
This is one of many, many, many possible strategies you can use! The key is to trade-off exploration vs exploitation
exploration: try something new you know little about and hope it is significantly better than your current best
exploitation: try to incrementally improve your current best solution
ax is one framework for handling this loop using various types of Gaussian Process models.
See also
https://ax.dev/docs/bayesopt.html
https://ax.dev/tutorials/
from ax.plot.trace import optimization_trace_single_method
from ax.service.managed_loop import optimize
from ax.utils.notebook.plotting import init_notebook_plotting, render
init_notebook_plotting()
[INFO 10-07 19:48:54] ax.utils.notebook.plotting: Injecting Plotly library into cell. Do not overwrite or delete cell.
from ax.service.ax_client import AxClient
def objective_ax(parameters):
x = parameters["x"]
noise = np.random.normal(loc=0, scale=0.05)
SEM = 0.05 / 2 # Assume that the value is +/- 0.05
return {"y": ((x**2 * np.sin(5 * np.pi * x) ** 6.0) + noise, SEM)}
ax_client = AxClient()
ax_client.create_experiment(
name="toy example",
parameters=[
{
"name": "x",
"type": "range",
"bounds": [0.0, 1.0],
},
],
objective_name="y",
minimize=False,
)
for i in range(50):
parameters, trial_index = ax_client.get_next_trial()
# Local evaluation here can be replaced with deployment to external system.
ax_client.complete_trial(trial_index=trial_index, raw_data=objective_ax(parameters))
[INFO 10-07 19:48:54] ax.service.ax_client: Starting optimization with verbose logging. To disable logging, set the `verbose_logging` argument to `False`. Note that float values in the logs are rounded to 6 decimal points.
[INFO 10-07 19:48:54] ax.service.utils.instantiation: Inferred value type of ParameterType.FLOAT for parameter x. If that is not the expected value type, you can explicity specify 'value_type' ('int', 'float', 'bool' or 'str') in parameter dict.
[INFO 10-07 19:48:54] ax.service.utils.instantiation: Created search space: SearchSpace(parameters=[RangeParameter(name='x', parameter_type=FLOAT, range=[0.0, 1.0])], parameter_constraints=[]).
[INFO 10-07 19:48:54] ax.modelbridge.dispatch_utils: Using Bayesian optimization since there are more ordered parameters than there are categories for the unordered categorical parameters.
[INFO 10-07 19:48:54] ax.modelbridge.dispatch_utils: Using Bayesian Optimization generation strategy: GenerationStrategy(name='Sobol+GPEI', steps=[Sobol for 5 trials, GPEI for subsequent trials]). Iterations after 5 will take longer to generate due to model-fitting.
[INFO 10-07 19:48:54] ax.service.ax_client: Generated new trial 0 with parameters {'x': 0.87064}.
[INFO 10-07 19:48:54] ax.service.ax_client: Completed trial 0 with data: {'y': (0.47455, 0.025)}.
[INFO 10-07 19:48:54] ax.service.ax_client: Generated new trial 1 with parameters {'x': 0.669018}.
[INFO 10-07 19:48:54] ax.service.ax_client: Completed trial 1 with data: {'y': (0.200453, 0.025)}.
[INFO 10-07 19:48:54] ax.service.ax_client: Generated new trial 2 with parameters {'x': 0.928723}.
[INFO 10-07 19:48:54] ax.service.ax_client: Completed trial 2 with data: {'y': (0.391691, 0.025)}.
[INFO 10-07 19:48:54] ax.service.ax_client: Generated new trial 3 with parameters {'x': 0.022226}.
[INFO 10-07 19:48:54] ax.service.ax_client: Completed trial 3 with data: {'y': (-0.032113, 0.025)}.
[INFO 10-07 19:48:54] ax.service.ax_client: Generated new trial 4 with parameters {'x': 0.574205}.
[INFO 10-07 19:48:54] ax.service.ax_client: Completed trial 4 with data: {'y': (0.05935, 0.025)}.
[INFO 10-07 19:48:54] ax.service.ax_client: Generated new trial 5 with parameters {'x': 0.832617}.
[INFO 10-07 19:48:54] ax.service.ax_client: Completed trial 5 with data: {'y': (0.05811, 0.025)}.
[INFO 10-07 19:48:55] ax.service.ax_client: Generated new trial 6 with parameters {'x': 0.891534}.
[INFO 10-07 19:48:55] ax.service.ax_client: Completed trial 6 with data: {'y': (0.786397, 0.025)}.
[INFO 10-07 19:48:55] ax.service.ax_client: Generated new trial 7 with parameters {'x': 0.898897}.
[INFO 10-07 19:48:55] ax.service.ax_client: Completed trial 7 with data: {'y': (0.727795, 0.025)}.
[INFO 10-07 19:48:55] ax.service.ax_client: Generated new trial 8 with parameters {'x': 0.304063}.
[INFO 10-07 19:48:55] ax.service.ax_client: Completed trial 8 with data: {'y': (0.044858, 0.025)}.
[INFO 10-07 19:48:56] ax.service.ax_client: Generated new trial 9 with parameters {'x': 0.440239}.
[INFO 10-07 19:48:56] ax.service.ax_client: Completed trial 9 with data: {'y': (-0.002949, 0.025)}.
[INFO 10-07 19:48:56] ax.service.ax_client: Generated new trial 10 with parameters {'x': 1.0}.
[INFO 10-07 19:48:56] ax.service.ax_client: Completed trial 10 with data: {'y': (-0.021265, 0.025)}.
[INFO 10-07 19:48:56] ax.service.ax_client: Generated new trial 11 with parameters {'x': 0.894309}.
[INFO 10-07 19:48:56] ax.service.ax_client: Completed trial 11 with data: {'y': (0.71591, 0.025)}.
[INFO 10-07 19:48:57] ax.service.ax_client: Generated new trial 12 with parameters {'x': 0.889265}.
[INFO 10-07 19:48:57] ax.service.ax_client: Completed trial 12 with data: {'y': (0.774117, 0.025)}.
[INFO 10-07 19:48:57] ax.service.ax_client: Generated new trial 13 with parameters {'x': 0.170235}.
[INFO 10-07 19:48:57] ax.service.ax_client: Completed trial 13 with data: {'y': (0.040335, 0.025)}.
[INFO 10-07 19:48:57] ax.service.ax_client: Generated new trial 14 with parameters {'x': 0.73436}.
[INFO 10-07 19:48:57] ax.service.ax_client: Completed trial 14 with data: {'y': (0.207792, 0.025)}.
[INFO 10-07 19:48:57] ax.service.ax_client: Generated new trial 15 with parameters {'x': 0.887704}.
[INFO 10-07 19:48:57] ax.service.ax_client: Completed trial 15 with data: {'y': (0.730615, 0.025)}.
[INFO 10-07 19:48:58] ax.service.ax_client: Generated new trial 16 with parameters {'x': 0.892761}.
[INFO 10-07 19:48:58] ax.service.ax_client: Completed trial 16 with data: {'y': (0.829079, 0.025)}.
[INFO 10-07 19:48:58] ax.service.ax_client: Generated new trial 17 with parameters {'x': 0.099447}.
[INFO 10-07 19:48:58] ax.service.ax_client: Completed trial 17 with data: {'y': (0.044162, 0.025)}.
[INFO 10-07 19:48:58] ax.service.ax_client: Generated new trial 18 with parameters {'x': 0.890576}.
[INFO 10-07 19:48:58] ax.service.ax_client: Completed trial 18 with data: {'y': (0.687407, 0.025)}.
[INFO 10-07 19:48:58] ax.service.ax_client: Generated new trial 19 with parameters {'x': 0.895907}.
[INFO 10-07 19:48:58] ax.service.ax_client: Completed trial 19 with data: {'y': (0.753267, 0.025)}.
[INFO 10-07 19:48:58] ax.service.ax_client: Generated new trial 20 with parameters {'x': 0.901035}.
[INFO 10-07 19:48:58] ax.service.ax_client: Completed trial 20 with data: {'y': (0.750588, 0.025)}.
[INFO 10-07 19:48:59] ax.service.ax_client: Generated new trial 21 with parameters {'x': 0.893538}.
[INFO 10-07 19:48:59] ax.service.ax_client: Completed trial 21 with data: {'y': (0.767749, 0.025)}.
[INFO 10-07 19:48:59] ax.service.ax_client: Generated new trial 22 with parameters {'x': 0.897246}.
[INFO 10-07 19:48:59] ax.service.ax_client: Completed trial 22 with data: {'y': (0.757545, 0.025)}.
[INFO 10-07 19:48:59] ax.service.ax_client: Generated new trial 23 with parameters {'x': 0.894969}.
[INFO 10-07 19:48:59] ax.service.ax_client: Completed trial 23 with data: {'y': (0.758109, 0.025)}.
[INFO 10-07 19:48:59] ax.service.ax_client: Generated new trial 24 with parameters {'x': 0.896491}.
[INFO 10-07 19:48:59] ax.service.ax_client: Completed trial 24 with data: {'y': (0.767037, 0.025)}.
[INFO 10-07 19:49:00] ax.service.ax_client: Generated new trial 25 with parameters {'x': 0.898107}.
[INFO 10-07 19:49:00] ax.service.ax_client: Completed trial 25 with data: {'y': (0.822505, 0.025)}.
[INFO 10-07 19:49:00] ax.service.ax_client: Generated new trial 26 with parameters {'x': 0.237314}.
[INFO 10-07 19:49:00] ax.service.ax_client: Completed trial 26 with data: {'y': (-0.153737, 0.025)}.
[INFO 10-07 19:49:00] ax.service.ax_client: Generated new trial 27 with parameters {'x': 0.369367}.
[INFO 10-07 19:49:00] ax.service.ax_client: Completed trial 27 with data: {'y': (0.007058, 0.025)}.
[INFO 10-07 19:49:00] ax.service.ax_client: Generated new trial 28 with parameters {'x': 0.904005}.
[INFO 10-07 19:49:00] ax.service.ax_client: Completed trial 28 with data: {'y': (0.755456, 0.025)}.
[INFO 10-07 19:49:00] ax.service.ax_client: Generated new trial 29 with parameters {'x': 0.899818}.
[INFO 10-07 19:49:00] ax.service.ax_client: Completed trial 29 with data: {'y': (0.797934, 0.025)}.
[INFO 10-07 19:49:01] ax.service.ax_client: Generated new trial 30 with parameters {'x': 0.897608}.
[INFO 10-07 19:49:01] ax.service.ax_client: Completed trial 30 with data: {'y': (0.757819, 0.025)}.
[INFO 10-07 19:49:01] ax.service.ax_client: Generated new trial 31 with parameters {'x': 0.895406}.
[INFO 10-07 19:49:01] ax.service.ax_client: Completed trial 31 with data: {'y': (0.792519, 0.025)}.
[INFO 10-07 19:49:01] ax.service.ax_client: Generated new trial 32 with parameters {'x': 0.89685}.
[INFO 10-07 19:49:01] ax.service.ax_client: Completed trial 32 with data: {'y': (0.797513, 0.025)}.
[INFO 10-07 19:49:01] ax.service.ax_client: Generated new trial 33 with parameters {'x': 0.902286}.
[INFO 10-07 19:49:01] ax.service.ax_client: Completed trial 33 with data: {'y': (0.72333, 0.025)}.
[INFO 10-07 19:49:02] ax.service.ax_client: Generated new trial 34 with parameters {'x': 0.509106}.
[INFO 10-07 19:49:02] ax.service.ax_client: Completed trial 34 with data: {'y': (0.163089, 0.025)}.
[INFO 10-07 19:49:02] ax.service.ax_client: Generated new trial 35 with parameters {'x': 0.896233}.
[INFO 10-07 19:49:02] ax.service.ax_client: Completed trial 35 with data: {'y': (0.784807, 0.025)}.
[INFO 10-07 19:49:02] ax.service.ax_client: Generated new trial 36 with parameters {'x': 0.894677}.
[INFO 10-07 19:49:02] ax.service.ax_client: Completed trial 36 with data: {'y': (0.777187, 0.025)}.
[INFO 10-07 19:49:02] ax.service.ax_client: Generated new trial 37 with parameters {'x': 0.895673}.
[INFO 10-07 19:49:02] ax.service.ax_client: Completed trial 37 with data: {'y': (0.753069, 0.025)}.
[INFO 10-07 19:49:03] ax.service.ax_client: Generated new trial 38 with parameters {'x': 0.89393}.
[INFO 10-07 19:49:03] ax.service.ax_client: Completed trial 38 with data: {'y': (0.75989, 0.025)}.
[INFO 10-07 19:49:03] ax.service.ax_client: Generated new trial 39 with parameters {'x': 0.897897}.
[INFO 10-07 19:49:03] ax.service.ax_client: Completed trial 39 with data: {'y': (0.798078, 0.025)}.
[INFO 10-07 19:49:03] ax.service.ax_client: Generated new trial 40 with parameters {'x': 0.899421}.
[INFO 10-07 19:49:03] ax.service.ax_client: Completed trial 40 with data: {'y': (0.713907, 0.025)}.
[INFO 10-07 19:49:03] ax.service.ax_client: Generated new trial 41 with parameters {'x': 0.897018}.
[INFO 10-07 19:49:03] ax.service.ax_client: Completed trial 41 with data: {'y': (0.91079, 0.025)}.
[INFO 10-07 19:49:04] ax.service.ax_client: Generated new trial 42 with parameters {'x': 0.898443}.
[INFO 10-07 19:49:04] ax.service.ax_client: Completed trial 42 with data: {'y': (0.822332, 0.025)}.
[INFO 10-07 19:49:04] ax.service.ax_client: Generated new trial 43 with parameters {'x': 0.896674}.
[INFO 10-07 19:49:04] ax.service.ax_client: Completed trial 43 with data: {'y': (0.820479, 0.025)}.
[INFO 10-07 19:49:04] ax.service.ax_client: Generated new trial 44 with parameters {'x': 0.89607}.
[INFO 10-07 19:49:04] ax.service.ax_client: Completed trial 44 with data: {'y': (0.770808, 0.025)}.
[INFO 10-07 19:49:05] ax.service.ax_client: Generated new trial 45 with parameters {'x': 0.624931}.
[INFO 10-07 19:49:05] ax.service.ax_client: Completed trial 45 with data: {'y': (0.030055, 0.025)}.
[INFO 10-07 19:49:05] ax.service.ax_client: Generated new trial 46 with parameters {'x': 0.895177}.
[INFO 10-07 19:49:05] ax.service.ax_client: Completed trial 46 with data: {'y': (0.911663, 0.025)}.
[INFO 10-07 19:49:05] ax.service.ax_client: Generated new trial 47 with parameters {'x': 0.895797}.
[INFO 10-07 19:49:05] ax.service.ax_client: Completed trial 47 with data: {'y': (0.773566, 0.025)}.
[INFO 10-07 19:49:05] ax.service.ax_client: Generated new trial 48 with parameters {'x': 0.892107}.
[INFO 10-07 19:49:05] ax.service.ax_client: Completed trial 48 with data: {'y': (0.729444, 0.025)}.
[INFO 10-07 19:49:06] ax.service.ax_client: Generated new trial 49 with parameters {'x': 0.902773}.
[INFO 10-07 19:49:06] ax.service.ax_client: Completed trial 49 with data: {'y': (0.784005, 0.025)}.
Let’s see where the points were sampled!
from ax.plot.slice import plot_slice
# Get the final model
model = ax_client.generation_strategy.model
# Make a slice plot (model vs x)
render(plot_slice(model, "x", "y", density=200))
import pandas as pd
# Make an empty dataframe!
df_ax = pd.DataFrame()
y_trials = []
for i in range(5):
ax_client = AxClient()
ax_client.create_experiment(
name="toy example",
parameters=[
{
"name": "x",
"type": "range",
"bounds": [0.0, 1.0],
},
],
objective_name="y",
minimize=False,
)
for i in range(100):
parameters, trial_index = ax_client.get_next_trial()
# Local evaluation here can be replaced with deployment to external system.
ax_client.complete_trial(
trial_index=trial_index, raw_data=objective_ax(parameters)
)
df_trials = ax_client.get_trials_data_frame()
y_trials.append(df_trials["y"].values)
[INFO 10-07 19:52:19] ax.service.ax_client: Starting optimization with verbose logging. To disable logging, set the `verbose_logging` argument to `False`. Note that float values in the logs are rounded to 6 decimal points.
[INFO 10-07 19:52:19] ax.service.utils.instantiation: Inferred value type of ParameterType.FLOAT for parameter x. If that is not the expected value type, you can explicity specify 'value_type' ('int', 'float', 'bool' or 'str') in parameter dict.
[INFO 10-07 19:52:19] ax.service.utils.instantiation: Created search space: SearchSpace(parameters=[RangeParameter(name='x', parameter_type=FLOAT, range=[0.0, 1.0])], parameter_constraints=[]).
[INFO 10-07 19:52:19] ax.modelbridge.dispatch_utils: Using Bayesian optimization since there are more ordered parameters than there are categories for the unordered categorical parameters.
[INFO 10-07 19:52:19] ax.modelbridge.dispatch_utils: Using Bayesian Optimization generation strategy: GenerationStrategy(name='Sobol+GPEI', steps=[Sobol for 5 trials, GPEI for subsequent trials]). Iterations after 5 will take longer to generate due to model-fitting.
[INFO 10-07 19:52:19] ax.service.ax_client: Generated new trial 0 with parameters {'x': 0.380537}.
[INFO 10-07 19:52:19] ax.service.ax_client: Completed trial 0 with data: {'y': (0.096838, 0.025)}.
[INFO 10-07 19:52:19] ax.service.ax_client: Generated new trial 1 with parameters {'x': 0.02521}.
[INFO 10-07 19:52:19] ax.service.ax_client: Completed trial 1 with data: {'y': (0.00298, 0.025)}.
[INFO 10-07 19:52:19] ax.service.ax_client: Generated new trial 2 with parameters {'x': 0.370598}.
[INFO 10-07 19:52:19] ax.service.ax_client: Completed trial 2 with data: {'y': (-0.056806, 0.025)}.
[INFO 10-07 19:52:19] ax.service.ax_client: Generated new trial 3 with parameters {'x': 0.363704}.
[INFO 10-07 19:52:19] ax.service.ax_client: Completed trial 3 with data: {'y': (-0.000187, 0.025)}.
[INFO 10-07 19:52:19] ax.service.ax_client: Generated new trial 4 with parameters {'x': 0.579913}.
[INFO 10-07 19:52:19] ax.service.ax_client: Completed trial 4 with data: {'y': (-0.02945, 0.025)}.
[INFO 10-07 19:52:19] ax.service.ax_client: Generated new trial 5 with parameters {'x': 1.0}.
[INFO 10-07 19:52:19] ax.service.ax_client: Completed trial 5 with data: {'y': (0.06696, 0.025)}.
[INFO 10-07 19:52:20] ax.service.ax_client: Generated new trial 6 with parameters {'x': 0.892472}.
[INFO 10-07 19:52:20] ax.service.ax_client: Completed trial 6 with data: {'y': (0.865666, 0.025)}.
[INFO 10-07 19:52:20] ax.service.ax_client: Generated new trial 7 with parameters {'x': 0.873358}.
[INFO 10-07 19:52:20] ax.service.ax_client: Completed trial 7 with data: {'y': (0.381059, 0.025)}.
[INFO 10-07 19:52:23] ax.service.ax_client: Generated new trial 8 with parameters {'x': 0.904768}.
[INFO 10-07 19:52:23] ax.service.ax_client: Completed trial 8 with data: {'y': (0.785596, 0.025)}.
[INFO 10-07 19:52:23] ax.service.ax_client: Generated new trial 9 with parameters {'x': 0.934854}.
[INFO 10-07 19:52:23] ax.service.ax_client: Completed trial 9 with data: {'y': (0.372067, 0.025)}.
[INFO 10-07 19:52:24] ax.service.ax_client: Generated new trial 10 with parameters {'x': 0.896448}.
[INFO 10-07 19:52:24] ax.service.ax_client: Completed trial 10 with data: {'y': (0.813174, 0.025)}.
[INFO 10-07 19:52:25] ax.service.ax_client: Generated new trial 11 with parameters {'x': 0.427287}.
[INFO 10-07 19:52:25] ax.service.ax_client: Completed trial 11 with data: {'y': (-0.082417, 0.025)}.
[INFO 10-07 19:52:25] ax.service.ax_client: Generated new trial 12 with parameters {'x': 0.304237}.
[INFO 10-07 19:52:25] ax.service.ax_client: Completed trial 12 with data: {'y': (0.07022, 0.025)}.
[INFO 10-07 19:52:25] ax.service.ax_client: Generated new trial 13 with parameters {'x': 0.168671}.
[INFO 10-07 19:52:25] ax.service.ax_client: Completed trial 13 with data: {'y': (0.030716, 0.025)}.
[INFO 10-07 19:52:26] ax.service.ax_client: Generated new trial 14 with parameters {'x': 0.725336}.
[INFO 10-07 19:52:26] ax.service.ax_client: Completed trial 14 with data: {'y': (0.367259, 0.025)}.
[INFO 10-07 19:52:26] ax.service.ax_client: Generated new trial 15 with parameters {'x': 0.889998}.
[INFO 10-07 19:52:26] ax.service.ax_client: Completed trial 15 with data: {'y': (0.761106, 0.025)}.
[INFO 10-07 19:52:26] ax.service.ax_client: Generated new trial 16 with parameters {'x': 0.898709}.
[INFO 10-07 19:52:26] ax.service.ax_client: Completed trial 16 with data: {'y': (0.825894, 0.025)}.
[INFO 10-07 19:52:27] ax.service.ax_client: Generated new trial 17 with parameters {'x': 0.763646}.
[INFO 10-07 19:52:27] ax.service.ax_client: Completed trial 17 with data: {'y': (-0.021781, 0.025)}.
[INFO 10-07 19:52:27] ax.service.ax_client: Generated new trial 18 with parameters {'x': 0.690741}.
[INFO 10-07 19:52:27] ax.service.ax_client: Completed trial 18 with data: {'y': (0.392919, 0.025)}.
[INFO 10-07 19:52:27] ax.service.ax_client: Generated new trial 19 with parameters {'x': 0.652862}.
[INFO 10-07 19:52:27] ax.service.ax_client: Completed trial 19 with data: {'y': (0.147739, 0.025)}.
[INFO 10-07 19:52:27] ax.service.ax_client: Generated new trial 20 with parameters {'x': 0.894948}.
[INFO 10-07 19:52:27] ax.service.ax_client: Completed trial 20 with data: {'y': (0.905171, 0.025)}.
[INFO 10-07 19:52:27] ax.service.ax_client: Generated new trial 21 with parameters {'x': 0.098126}.
[INFO 10-07 19:52:27] ax.service.ax_client: Completed trial 21 with data: {'y': (-0.070187, 0.025)}.
[INFO 10-07 19:52:28] ax.service.ax_client: Generated new trial 22 with parameters {'x': 0.897379}.
[INFO 10-07 19:52:28] ax.service.ax_client: Completed trial 22 with data: {'y': (0.803711, 0.025)}.
[INFO 10-07 19:52:28] ax.service.ax_client: Generated new trial 23 with parameters {'x': 0.893939}.
[INFO 10-07 19:52:28] ax.service.ax_client: Completed trial 23 with data: {'y': (0.783935, 0.025)}.
[INFO 10-07 19:52:28] ax.service.ax_client: Generated new trial 24 with parameters {'x': 0.912953}.
[INFO 10-07 19:52:28] ax.service.ax_client: Completed trial 24 with data: {'y': (0.860496, 0.025)}.
[INFO 10-07 19:52:29] ax.service.ax_client: Generated new trial 25 with parameters {'x': 0.916825}.
[INFO 10-07 19:52:29] ax.service.ax_client: Completed trial 25 with data: {'y': (0.734746, 0.025)}.
[INFO 10-07 19:52:30] ax.service.ax_client: Generated new trial 26 with parameters {'x': 0.910327}.
[INFO 10-07 19:52:30] ax.service.ax_client: Completed trial 26 with data: {'y': (0.669313, 0.025)}.
[INFO 10-07 19:52:30] ax.service.ax_client: Generated new trial 27 with parameters {'x': 0.901633}.
[INFO 10-07 19:52:30] ax.service.ax_client: Completed trial 27 with data: {'y': (0.822476, 0.025)}.
[INFO 10-07 19:52:31] ax.service.ax_client: Generated new trial 28 with parameters {'x': 0.677557}.
[INFO 10-07 19:52:31] ax.service.ax_client: Completed trial 28 with data: {'y': (0.330106, 0.025)}.
[INFO 10-07 19:52:32] ax.service.ax_client: Generated new trial 29 with parameters {'x': 0.914098}.
[INFO 10-07 19:52:32] ax.service.ax_client: Completed trial 29 with data: {'y': (0.718655, 0.025)}.
[INFO 10-07 19:52:32] ax.service.ax_client: Generated new trial 30 with parameters {'x': 0.238153}.
[INFO 10-07 19:52:32] ax.service.ax_client: Completed trial 30 with data: {'y': (0.05824, 0.025)}.
[INFO 10-07 19:52:33] ax.service.ax_client: Generated new trial 31 with parameters {'x': 0.506444}.
[INFO 10-07 19:52:33] ax.service.ax_client: Completed trial 31 with data: {'y': (0.151056, 0.025)}.
[INFO 10-07 19:52:33] ax.service.ax_client: Generated new trial 32 with parameters {'x': 0.895704}.
[INFO 10-07 19:52:33] ax.service.ax_client: Completed trial 32 with data: {'y': (0.90437, 0.025)}.
[INFO 10-07 19:52:33] ax.service.ax_client: Generated new trial 33 with parameters {'x': 0.82133}.
[INFO 10-07 19:52:33] ax.service.ax_client: Completed trial 33 with data: {'y': (-0.01729, 0.025)}.
[INFO 10-07 19:52:34] ax.service.ax_client: Generated new trial 34 with parameters {'x': 0.920107}.
[INFO 10-07 19:52:34] ax.service.ax_client: Completed trial 34 with data: {'y': (0.499795, 0.025)}.
[INFO 10-07 19:52:35] ax.service.ax_client: Generated new trial 35 with parameters {'x': 0.891561}.
[INFO 10-07 19:52:35] ax.service.ax_client: Completed trial 35 with data: {'y': (0.854825, 0.025)}.
[INFO 10-07 19:52:35] ax.service.ax_client: Generated new trial 36 with parameters {'x': 0.903151}.
[INFO 10-07 19:52:35] ax.service.ax_client: Completed trial 36 with data: {'y': (0.781117, 0.025)}.
[INFO 10-07 19:52:36] ax.service.ax_client: Generated new trial 37 with parameters {'x': 0.900215}.
[INFO 10-07 19:52:36] ax.service.ax_client: Completed trial 37 with data: {'y': (0.771982, 0.025)}.
[INFO 10-07 19:52:36] ax.service.ax_client: Generated new trial 38 with parameters {'x': 0.708224}.
[INFO 10-07 19:52:36] ax.service.ax_client: Completed trial 38 with data: {'y': (0.462884, 0.025)}.
[INFO 10-07 19:52:36] ax.service.ax_client: Generated new trial 39 with parameters {'x': 0.95147}.
[INFO 10-07 19:52:36] ax.service.ax_client: Completed trial 39 with data: {'y': (0.116843, 0.025)}.
[INFO 10-07 19:52:37] ax.service.ax_client: Generated new trial 40 with parameters {'x': 0.893322}.
[INFO 10-07 19:52:37] ax.service.ax_client: Completed trial 40 with data: {'y': (0.816022, 0.025)}.
[INFO 10-07 19:52:37] ax.service.ax_client: Generated new trial 41 with parameters {'x': 0.912302}.
[INFO 10-07 19:52:37] ax.service.ax_client: Completed trial 41 with data: {'y': (0.74528, 0.025)}.
[INFO 10-07 19:52:37] ax.service.ax_client: Generated new trial 42 with parameters {'x': 0.853754}.
[INFO 10-07 19:52:37] ax.service.ax_client: Completed trial 42 with data: {'y': (0.182905, 0.025)}.
[INFO 10-07 19:52:38] ax.service.ax_client: Generated new trial 43 with parameters {'x': 0.884038}.
[INFO 10-07 19:52:38] ax.service.ax_client: Completed trial 43 with data: {'y': (0.644576, 0.025)}.
[INFO 10-07 19:52:38] ax.service.ax_client: Generated new trial 44 with parameters {'x': 0.473671}.
[INFO 10-07 19:52:38] ax.service.ax_client: Completed trial 44 with data: {'y': (0.097415, 0.025)}.
[INFO 10-07 19:52:38] ax.service.ax_client: Generated new trial 45 with parameters {'x': 0.537314}.
[INFO 10-07 19:52:38] ax.service.ax_client: Completed trial 45 with data: {'y': (0.062334, 0.025)}.
[INFO 10-07 19:52:38] ax.service.ax_client: Generated new trial 46 with parameters {'x': 0.894502}.
[INFO 10-07 19:52:38] ax.service.ax_client: Completed trial 46 with data: {'y': (0.741592, 0.025)}.
[INFO 10-07 19:52:39] ax.service.ax_client: Generated new trial 47 with parameters {'x': 0.896917}.
[INFO 10-07 19:52:39] ax.service.ax_client: Completed trial 47 with data: {'y': (0.767678, 0.025)}.
[INFO 10-07 19:52:39] ax.service.ax_client: Generated new trial 48 with parameters {'x': 0.621104}.
[INFO 10-07 19:52:39] ax.service.ax_client: Completed trial 48 with data: {'y': (-0.018097, 0.025)}.
[INFO 10-07 19:52:39] ax.service.ax_client: Generated new trial 49 with parameters {'x': 0.892934}.
[INFO 10-07 19:52:39] ax.service.ax_client: Completed trial 49 with data: {'y': (0.76339, 0.025)}.
[INFO 10-07 19:52:40] ax.service.ax_client: Generated new trial 50 with parameters {'x': 0.895359}.
[INFO 10-07 19:52:40] ax.service.ax_client: Completed trial 50 with data: {'y': (0.695044, 0.025)}.
[INFO 10-07 19:52:40] ax.service.ax_client: Generated new trial 51 with parameters {'x': 0.899457}.
[INFO 10-07 19:52:40] ax.service.ax_client: Completed trial 51 with data: {'y': (0.791614, 0.025)}.
[INFO 10-07 19:52:40] ax.service.ax_client: Generated new trial 52 with parameters {'x': 0.204115}.
[INFO 10-07 19:52:40] ax.service.ax_client: Completed trial 52 with data: {'y': (-0.084157, 0.025)}.
[INFO 10-07 19:52:41] ax.service.ax_client: Generated new trial 53 with parameters {'x': 0.271319}.
[INFO 10-07 19:52:41] ax.service.ax_client: Completed trial 53 with data: {'y': (0.026436, 0.025)}.
[INFO 10-07 19:52:41] ax.service.ax_client: Generated new trial 54 with parameters {'x': 0.892013}.
[INFO 10-07 19:52:41] ax.service.ax_client: Completed trial 54 with data: {'y': (0.722362, 0.025)}.
[INFO 10-07 19:52:41] ax.service.ax_client: Generated new trial 55 with parameters {'x': 0.333995}.
[INFO 10-07 19:52:41] ax.service.ax_client: Completed trial 55 with data: {'y': (0.104689, 0.025)}.
[INFO 10-07 19:52:41] ax.service.ax_client: Generated new trial 56 with parameters {'x': 0.898083}.
[INFO 10-07 19:52:41] ax.service.ax_client: Completed trial 56 with data: {'y': (0.791022, 0.025)}.
[INFO 10-07 19:52:42] ax.service.ax_client: Generated new trial 57 with parameters {'x': 0.900931}.
[INFO 10-07 19:52:42] ax.service.ax_client: Completed trial 57 with data: {'y': (0.876464, 0.025)}.
[INFO 10-07 19:52:42] ax.service.ax_client: Generated new trial 58 with parameters {'x': 0.060292}.
[INFO 10-07 19:52:42] ax.service.ax_client: Completed trial 58 with data: {'y': (-0.064418, 0.025)}.
[INFO 10-07 19:52:42] ax.service.ax_client: Generated new trial 59 with parameters {'x': 0.13549}.
[INFO 10-07 19:52:42] ax.service.ax_client: Completed trial 59 with data: {'y': (-0.001025, 0.025)}.
[INFO 10-07 19:52:43] ax.service.ax_client: Generated new trial 60 with parameters {'x': 0.90225}.
[INFO 10-07 19:52:43] ax.service.ax_client: Completed trial 60 with data: {'y': (0.745662, 0.025)}.
[INFO 10-07 19:52:43] ax.service.ax_client: Generated new trial 61 with parameters {'x': 0.900641}.
[INFO 10-07 19:52:43] ax.service.ax_client: Completed trial 61 with data: {'y': (0.8175, 0.025)}.
[INFO 10-07 19:52:43] ax.service.ax_client: Generated new trial 62 with parameters {'x': 0.899816}.
[INFO 10-07 19:52:43] ax.service.ax_client: Completed trial 62 with data: {'y': (0.818872, 0.025)}.
[INFO 10-07 19:52:44] ax.service.ax_client: Generated new trial 63 with parameters {'x': 0.89913}.
[INFO 10-07 19:52:44] ax.service.ax_client: Completed trial 63 with data: {'y': (0.791648, 0.025)}.
[INFO 10-07 19:52:44] ax.service.ax_client: Generated new trial 64 with parameters {'x': 0.890878}.
[INFO 10-07 19:52:44] ax.service.ax_client: Completed trial 64 with data: {'y': (0.802397, 0.025)}.
[INFO 10-07 19:52:45] ax.service.ax_client: Generated new trial 65 with parameters {'x': 0.896074}.
[INFO 10-07 19:52:45] ax.service.ax_client: Completed trial 65 with data: {'y': (0.813222, 0.025)}.
[INFO 10-07 19:52:45] ax.service.ax_client: Generated new trial 66 with parameters {'x': 0.901281}.
[INFO 10-07 19:52:45] ax.service.ax_client: Completed trial 66 with data: {'y': (0.828776, 0.025)}.
[INFO 10-07 19:52:45] ax.service.ax_client: Generated new trial 67 with parameters {'x': 0.900392}.
[INFO 10-07 19:52:45] ax.service.ax_client: Completed trial 67 with data: {'y': (0.807892, 0.025)}.
[INFO 10-07 19:52:46] ax.service.ax_client: Generated new trial 68 with parameters {'x': 0.898386}.
[INFO 10-07 19:52:46] ax.service.ax_client: Completed trial 68 with data: {'y': (0.843326, 0.025)}.
[INFO 10-07 19:52:46] ax.service.ax_client: Generated new trial 69 with parameters {'x': 0.897685}.
[INFO 10-07 19:52:46] ax.service.ax_client: Completed trial 69 with data: {'y': (0.729714, 0.025)}.
[INFO 10-07 19:52:46] ax.service.ax_client: Generated new trial 70 with parameters {'x': 0.904084}.
[INFO 10-07 19:52:46] ax.service.ax_client: Completed trial 70 with data: {'y': (0.867666, 0.025)}.
[INFO 10-07 19:52:47] ax.service.ax_client: Generated new trial 71 with parameters {'x': 0.902724}.
[INFO 10-07 19:52:47] ax.service.ax_client: Completed trial 71 with data: {'y': (0.850141, 0.025)}.
[INFO 10-07 19:52:47] ax.service.ax_client: Generated new trial 72 with parameters {'x': 0.901902}.
[INFO 10-07 19:52:47] ax.service.ax_client: Completed trial 72 with data: {'y': (0.827499, 0.025)}.
[INFO 10-07 19:52:48] ax.service.ax_client: Generated new trial 73 with parameters {'x': 0.903534}.
[INFO 10-07 19:52:48] ax.service.ax_client: Completed trial 73 with data: {'y': (0.816088, 0.025)}.
[INFO 10-07 19:52:48] ax.service.ax_client: Generated new trial 74 with parameters {'x': 0.90559}.
[INFO 10-07 19:52:48] ax.service.ax_client: Completed trial 74 with data: {'y': (0.812557, 0.025)}.
[INFO 10-07 19:52:49] ax.service.ax_client: Generated new trial 75 with parameters {'x': 0.902508}.
[INFO 10-07 19:52:49] ax.service.ax_client: Completed trial 75 with data: {'y': (0.798839, 0.025)}.
[INFO 10-07 19:52:49] ax.service.ax_client: Generated new trial 76 with parameters {'x': 0.0}.
[INFO 10-07 19:52:49] ax.service.ax_client: Completed trial 76 with data: {'y': (-0.023519, 0.025)}.
[INFO 10-07 19:52:50] ax.service.ax_client: Generated new trial 77 with parameters {'x': 0.906509}.
[INFO 10-07 19:52:50] ax.service.ax_client: Completed trial 77 with data: {'y': (0.674439, 0.025)}.
[INFO 10-07 19:52:50] ax.service.ax_client: Generated new trial 78 with parameters {'x': 0.902075}.
[INFO 10-07 19:52:50] ax.service.ax_client: Completed trial 78 with data: {'y': (0.871295, 0.025)}.
[INFO 10-07 19:52:51] ax.service.ax_client: Generated new trial 79 with parameters {'x': 0.902914}.
[INFO 10-07 19:52:51] ax.service.ax_client: Completed trial 79 with data: {'y': (0.751245, 0.025)}.
[INFO 10-07 19:52:51] ax.service.ax_client: Generated new trial 80 with parameters {'x': 0.901471}.
[INFO 10-07 19:52:51] ax.service.ax_client: Completed trial 80 with data: {'y': (0.856003, 0.025)}.
[INFO 10-07 19:52:52] ax.service.ax_client: Generated new trial 81 with parameters {'x': 0.88874}.
[INFO 10-07 19:52:52] ax.service.ax_client: Completed trial 81 with data: {'y': (0.670526, 0.025)}.
[INFO 10-07 19:52:52] ax.service.ax_client: Generated new trial 82 with parameters {'x': 0.739297}.
[INFO 10-07 19:52:52] ax.service.ax_client: Completed trial 82 with data: {'y': (0.205882, 0.025)}.
[INFO 10-07 19:52:53] ax.service.ax_client: Generated new trial 83 with parameters {'x': 0.894268}.
[INFO 10-07 19:52:53] ax.service.ax_client: Completed trial 83 with data: {'y': (0.696453, 0.025)}.
[INFO 10-07 19:52:53] ax.service.ax_client: Generated new trial 84 with parameters {'x': 0.699999}.
[INFO 10-07 19:52:53] ax.service.ax_client: Completed trial 84 with data: {'y': (0.522022, 0.025)}.
[INFO 10-07 19:52:53] ax.service.ax_client: Generated new trial 85 with parameters {'x': 0.901763}.
[INFO 10-07 19:52:53] ax.service.ax_client: Completed trial 85 with data: {'y': (0.811655, 0.025)}.
[INFO 10-07 19:52:54] ax.service.ax_client: Generated new trial 86 with parameters {'x': 0.903832}.
[INFO 10-07 19:52:54] ax.service.ax_client: Completed trial 86 with data: {'y': (0.861917, 0.025)}.
[INFO 10-07 19:52:54] ax.service.ax_client: Generated new trial 87 with parameters {'x': 0.79259}.
[INFO 10-07 19:52:54] ax.service.ax_client: Completed trial 87 with data: {'y': (-0.049067, 0.025)}.
[INFO 10-07 19:52:55] ax.service.ax_client: Generated new trial 88 with parameters {'x': 0.901112}.
[INFO 10-07 19:52:55] ax.service.ax_client: Completed trial 88 with data: {'y': (0.824497, 0.025)}.
[INFO 10-07 19:52:55] ax.service.ax_client: Generated new trial 89 with parameters {'x': 0.975738}.
[INFO 10-07 19:52:55] ax.service.ax_client: Completed trial 89 with data: {'y': (0.03835, 0.025)}.
[INFO 10-07 19:52:56] ax.service.ax_client: Generated new trial 90 with parameters {'x': 0.902617}.
[INFO 10-07 19:52:56] ax.service.ax_client: Completed trial 90 with data: {'y': (0.790366, 0.025)}.
[INFO 10-07 19:52:56] ax.service.ax_client: Generated new trial 91 with parameters {'x': 0.900774}.
[INFO 10-07 19:52:56] ax.service.ax_client: Completed trial 91 with data: {'y': (0.785376, 0.025)}.
[INFO 10-07 19:52:57] ax.service.ax_client: Generated new trial 92 with parameters {'x': 0.400147}.
[INFO 10-07 19:52:57] ax.service.ax_client: Completed trial 92 with data: {'y': (-0.063736, 0.025)}.
[INFO 10-07 19:52:58] ax.service.ax_client: Generated new trial 93 with parameters {'x': 0.903338}.
[INFO 10-07 19:52:58] ax.service.ax_client: Completed trial 93 with data: {'y': (0.853248, 0.025)}.
[INFO 10-07 19:52:58] ax.service.ax_client: Generated new trial 94 with parameters {'x': 0.452973}.
[INFO 10-07 19:52:58] ax.service.ax_client: Completed trial 94 with data: {'y': (0.015872, 0.025)}.
[INFO 10-07 19:52:59] ax.service.ax_client: Generated new trial 95 with parameters {'x': 0.902379}.
[INFO 10-07 19:52:59] ax.service.ax_client: Completed trial 95 with data: {'y': (0.872978, 0.025)}.
[INFO 10-07 19:52:59] ax.service.ax_client: Generated new trial 96 with parameters {'x': 0.902168}.
[INFO 10-07 19:52:59] ax.service.ax_client: Completed trial 96 with data: {'y': (0.871265, 0.025)}.
[INFO 10-07 19:53:00] ax.service.ax_client: Generated new trial 97 with parameters {'x': 0.895784}.
[INFO 10-07 19:53:00] ax.service.ax_client: Completed trial 97 with data: {'y': (0.837295, 0.025)}.
[INFO 10-07 19:53:00] ax.service.ax_client: Generated new trial 98 with parameters {'x': 0.901984}.
[INFO 10-07 19:53:00] ax.service.ax_client: Completed trial 98 with data: {'y': (0.753965, 0.025)}.
[INFO 10-07 19:53:01] ax.service.ax_client: Generated new trial 99 with parameters {'x': 0.902822}.
[INFO 10-07 19:53:01] ax.service.ax_client: Completed trial 99 with data: {'y': (0.775514, 0.025)}.
[INFO 10-07 19:53:01] ax.service.ax_client: Starting optimization with verbose logging. To disable logging, set the `verbose_logging` argument to `False`. Note that float values in the logs are rounded to 6 decimal points.
[INFO 10-07 19:53:01] ax.service.utils.instantiation: Inferred value type of ParameterType.FLOAT for parameter x. If that is not the expected value type, you can explicity specify 'value_type' ('int', 'float', 'bool' or 'str') in parameter dict.
[INFO 10-07 19:53:01] ax.service.utils.instantiation: Created search space: SearchSpace(parameters=[RangeParameter(name='x', parameter_type=FLOAT, range=[0.0, 1.0])], parameter_constraints=[]).
[INFO 10-07 19:53:01] ax.modelbridge.dispatch_utils: Using Bayesian optimization since there are more ordered parameters than there are categories for the unordered categorical parameters.
[INFO 10-07 19:53:01] ax.modelbridge.dispatch_utils: Using Bayesian Optimization generation strategy: GenerationStrategy(name='Sobol+GPEI', steps=[Sobol for 5 trials, GPEI for subsequent trials]). Iterations after 5 will take longer to generate due to model-fitting.
[INFO 10-07 19:53:01] ax.service.ax_client: Generated new trial 0 with parameters {'x': 0.405639}.
[INFO 10-07 19:53:01] ax.service.ax_client: Completed trial 0 with data: {'y': (-0.006165, 0.025)}.
[INFO 10-07 19:53:01] ax.service.ax_client: Generated new trial 1 with parameters {'x': 0.092996}.
[INFO 10-07 19:53:01] ax.service.ax_client: Completed trial 1 with data: {'y': (0.017296, 0.025)}.
[INFO 10-07 19:53:01] ax.service.ax_client: Generated new trial 2 with parameters {'x': 0.134999}.
[INFO 10-07 19:53:01] ax.service.ax_client: Completed trial 2 with data: {'y': (-3.9e-05, 0.025)}.
[INFO 10-07 19:53:01] ax.service.ax_client: Generated new trial 3 with parameters {'x': 0.229527}.
[INFO 10-07 19:53:01] ax.service.ax_client: Completed trial 3 with data: {'y': (-0.021107, 0.025)}.
[INFO 10-07 19:53:01] ax.service.ax_client: Generated new trial 4 with parameters {'x': 0.591379}.
[INFO 10-07 19:53:01] ax.service.ax_client: Completed trial 4 with data: {'y': (-0.090603, 0.025)}.
[INFO 10-07 19:53:01] ax.service.ax_client: Generated new trial 5 with parameters {'x': 0.0}.
[INFO 10-07 19:53:01] ax.service.ax_client: Completed trial 5 with data: {'y': (0.021334, 0.025)}.
[INFO 10-07 19:53:01] ax.service.ax_client: Generated new trial 6 with parameters {'x': 1.0}.
[INFO 10-07 19:53:01] ax.service.ax_client: Completed trial 6 with data: {'y': (0.017122, 0.025)}.
[INFO 10-07 19:53:02] ax.service.ax_client: Generated new trial 7 with parameters {'x': 0.899226}.
[INFO 10-07 19:53:02] ax.service.ax_client: Completed trial 7 with data: {'y': (0.75638, 0.025)}.
[INFO 10-07 19:53:02] ax.service.ax_client: Generated new trial 8 with parameters {'x': 0.840845}.
[INFO 10-07 19:53:02] ax.service.ax_client: Completed trial 8 with data: {'y': (0.01245, 0.025)}.
[INFO 10-07 19:53:02] ax.service.ax_client: Generated new trial 9 with parameters {'x': 0.914452}.
[INFO 10-07 19:53:02] ax.service.ax_client: Completed trial 9 with data: {'y': (0.753332, 0.025)}.
[INFO 10-07 19:53:02] ax.service.ax_client: Generated new trial 10 with parameters {'x': 0.906749}.
[INFO 10-07 19:53:02] ax.service.ax_client: Completed trial 10 with data: {'y': (0.766574, 0.025)}.
[INFO 10-07 19:53:03] ax.service.ax_client: Generated new trial 11 with parameters {'x': 0.931367}.
[INFO 10-07 19:53:03] ax.service.ax_client: Completed trial 11 with data: {'y': (0.357974, 0.025)}.
[INFO 10-07 19:53:03] ax.service.ax_client: Generated new trial 12 with parameters {'x': 0.717484}.
[INFO 10-07 19:53:03] ax.service.ax_client: Completed trial 12 with data: {'y': (0.421013, 0.025)}.
[INFO 10-07 19:53:03] ax.service.ax_client: Generated new trial 13 with parameters {'x': 0.679866}.
[INFO 10-07 19:53:03] ax.service.ax_client: Completed trial 13 with data: {'y': (0.364201, 0.025)}.
[INFO 10-07 19:53:03] ax.service.ax_client: Generated new trial 14 with parameters {'x': 0.759149}.
[INFO 10-07 19:53:03] ax.service.ax_client: Completed trial 14 with data: {'y': (-0.026838, 0.025)}.
[INFO 10-07 19:53:03] ax.service.ax_client: Generated new trial 15 with parameters {'x': 0.887211}.
[INFO 10-07 19:53:03] ax.service.ax_client: Completed trial 15 with data: {'y': (0.706498, 0.025)}.
[INFO 10-07 19:53:04] ax.service.ax_client: Generated new trial 16 with parameters {'x': 0.493837}.
[INFO 10-07 19:53:04] ax.service.ax_client: Completed trial 16 with data: {'y': (0.203819, 0.025)}.
[INFO 10-07 19:53:04] ax.service.ax_client: Generated new trial 17 with parameters {'x': 0.909287}.
[INFO 10-07 19:53:04] ax.service.ax_client: Completed trial 17 with data: {'y': (0.711722, 0.025)}.
[INFO 10-07 19:53:04] ax.service.ax_client: Generated new trial 18 with parameters {'x': 0.903089}.
[INFO 10-07 19:53:04] ax.service.ax_client: Completed trial 18 with data: {'y': (0.826619, 0.025)}.
[INFO 10-07 19:53:05] ax.service.ax_client: Generated new trial 19 with parameters {'x': 0.901379}.
[INFO 10-07 19:53:05] ax.service.ax_client: Completed trial 19 with data: {'y': (0.794491, 0.025)}.
[INFO 10-07 19:53:05] ax.service.ax_client: Generated new trial 20 with parameters {'x': 0.317744}.
[INFO 10-07 19:53:05] ax.service.ax_client: Completed trial 20 with data: {'y': (0.057301, 0.025)}.
[INFO 10-07 19:53:05] ax.service.ax_client: Generated new trial 21 with parameters {'x': 0.536387}.
[INFO 10-07 19:53:05] ax.service.ax_client: Completed trial 21 with data: {'y': (0.078223, 0.025)}.
[INFO 10-07 19:53:05] ax.service.ax_client: Generated new trial 22 with parameters {'x': 0.896685}.
[INFO 10-07 19:53:05] ax.service.ax_client: Completed trial 22 with data: {'y': (0.790676, 0.025)}.
[INFO 10-07 19:53:05] ax.service.ax_client: Generated new trial 23 with parameters {'x': 0.904687}.
[INFO 10-07 19:53:05] ax.service.ax_client: Completed trial 23 with data: {'y': (0.706936, 0.025)}.
[INFO 10-07 19:53:05] ax.service.ax_client: Generated new trial 24 with parameters {'x': 0.894733}.
[INFO 10-07 19:53:05] ax.service.ax_client: Completed trial 24 with data: {'y': (0.7294, 0.025)}.
[INFO 10-07 19:53:06] ax.service.ax_client: Generated new trial 25 with parameters {'x': 0.900297}.
[INFO 10-07 19:53:06] ax.service.ax_client: Completed trial 25 with data: {'y': (0.804969, 0.025)}.
[INFO 10-07 19:53:06] ax.service.ax_client: Generated new trial 26 with parameters {'x': 0.643741}.
[INFO 10-07 19:53:06] ax.service.ax_client: Completed trial 26 with data: {'y': (-0.013217, 0.025)}.
[INFO 10-07 19:53:06] ax.service.ax_client: Generated new trial 27 with parameters {'x': 0.897958}.
[INFO 10-07 19:53:06] ax.service.ax_client: Completed trial 27 with data: {'y': (0.838577, 0.025)}.
[INFO 10-07 19:53:07] ax.service.ax_client: Generated new trial 28 with parameters {'x': 0.898534}.
[INFO 10-07 19:53:07] ax.service.ax_client: Completed trial 28 with data: {'y': (0.845751, 0.025)}.
[INFO 10-07 19:53:07] ax.service.ax_client: Generated new trial 29 with parameters {'x': 0.91284}.
[INFO 10-07 19:53:07] ax.service.ax_client: Completed trial 29 with data: {'y': (0.728919, 0.025)}.
[INFO 10-07 19:53:07] ax.service.ax_client: Generated new trial 30 with parameters {'x': 0.899687}.
[INFO 10-07 19:53:07] ax.service.ax_client: Completed trial 30 with data: {'y': (0.725954, 0.025)}.
[INFO 10-07 19:53:07] ax.service.ax_client: Generated new trial 31 with parameters {'x': 0.902085}.
[INFO 10-07 19:53:07] ax.service.ax_client: Completed trial 31 with data: {'y': (0.807152, 0.025)}.
[INFO 10-07 19:53:08] ax.service.ax_client: Generated new trial 32 with parameters {'x': 0.90089}.
[INFO 10-07 19:53:08] ax.service.ax_client: Completed trial 32 with data: {'y': (0.767833, 0.025)}.
[INFO 10-07 19:53:08] ax.service.ax_client: Generated new trial 33 with parameters {'x': 0.897449}.
[INFO 10-07 19:53:08] ax.service.ax_client: Completed trial 33 with data: {'y': (0.746888, 0.025)}.
[INFO 10-07 19:53:08] ax.service.ax_client: Generated new trial 34 with parameters {'x': 0.903717}.
[INFO 10-07 19:53:08] ax.service.ax_client: Completed trial 34 with data: {'y': (0.81625, 0.025)}.
[INFO 10-07 19:53:08] ax.service.ax_client: Generated new trial 35 with parameters {'x': 0.902528}.
[INFO 10-07 19:53:08] ax.service.ax_client: Completed trial 35 with data: {'y': (0.783058, 0.025)}.
[INFO 10-07 19:53:09] ax.service.ax_client: Generated new trial 36 with parameters {'x': 0.90176}.
[INFO 10-07 19:53:09] ax.service.ax_client: Completed trial 36 with data: {'y': (0.868821, 0.025)}.
[INFO 10-07 19:53:09] ax.service.ax_client: Generated new trial 37 with parameters {'x': 0.905593}.
[INFO 10-07 19:53:09] ax.service.ax_client: Completed trial 37 with data: {'y': (0.811756, 0.025)}.
[INFO 10-07 19:53:09] ax.service.ax_client: Generated new trial 38 with parameters {'x': 0.900594}.
[INFO 10-07 19:53:09] ax.service.ax_client: Completed trial 38 with data: {'y': (0.851287, 0.025)}.
[INFO 10-07 19:53:09] ax.service.ax_client: Generated new trial 39 with parameters {'x': 0.900003}.
[INFO 10-07 19:53:09] ax.service.ax_client: Completed trial 39 with data: {'y': (0.831675, 0.025)}.
[INFO 10-07 19:53:10] ax.service.ax_client: Generated new trial 40 with parameters {'x': 0.901142}.
[INFO 10-07 19:53:10] ax.service.ax_client: Completed trial 40 with data: {'y': (0.835044, 0.025)}.
[INFO 10-07 19:53:10] ax.service.ax_client: Generated new trial 41 with parameters {'x': 0.902287}.
[INFO 10-07 19:53:10] ax.service.ax_client: Completed trial 41 with data: {'y': (0.791112, 0.025)}.
[INFO 10-07 19:53:10] ax.service.ax_client: Generated new trial 42 with parameters {'x': 0.901601}.
[INFO 10-07 19:53:10] ax.service.ax_client: Completed trial 42 with data: {'y': (0.797294, 0.025)}.
[INFO 10-07 19:53:11] ax.service.ax_client: Generated new trial 43 with parameters {'x': 0.453906}.
[INFO 10-07 19:53:11] ax.service.ax_client: Completed trial 43 with data: {'y': (0.037028, 0.025)}.
[INFO 10-07 19:53:11] ax.service.ax_client: Generated new trial 44 with parameters {'x': 0.902764}.
[INFO 10-07 19:53:11] ax.service.ax_client: Completed trial 44 with data: {'y': (0.872071, 0.025)}.
[INFO 10-07 19:53:11] ax.service.ax_client: Generated new trial 45 with parameters {'x': 0.046091}.
[INFO 10-07 19:53:11] ax.service.ax_client: Completed trial 45 with data: {'y': (-0.015712, 0.025)}.
[INFO 10-07 19:53:11] ax.service.ax_client: Generated new trial 46 with parameters {'x': 0.901924}.
[INFO 10-07 19:53:11] ax.service.ax_client: Completed trial 46 with data: {'y': (0.793544, 0.025)}.
[INFO 10-07 19:53:11] ax.service.ax_client: Generated new trial 47 with parameters {'x': 0.903404}.
[INFO 10-07 19:53:11] ax.service.ax_client: Completed trial 47 with data: {'y': (0.873993, 0.025)}.
[INFO 10-07 19:53:12] ax.service.ax_client: Generated new trial 48 with parameters {'x': 0.904169}.
[INFO 10-07 19:53:12] ax.service.ax_client: Completed trial 48 with data: {'y': (0.781498, 0.025)}.
[INFO 10-07 19:53:12] ax.service.ax_client: Generated new trial 49 with parameters {'x': 0.182479}.
[INFO 10-07 19:53:12] ax.service.ax_client: Completed trial 49 with data: {'y': (0.075985, 0.025)}.
[INFO 10-07 19:53:12] ax.service.ax_client: Generated new trial 50 with parameters {'x': 0.902418}.
[INFO 10-07 19:53:12] ax.service.ax_client: Completed trial 50 with data: {'y': (0.766827, 0.025)}.
[INFO 10-07 19:53:12] ax.service.ax_client: Generated new trial 51 with parameters {'x': 0.902931}.
[INFO 10-07 19:53:12] ax.service.ax_client: Completed trial 51 with data: {'y': (0.839725, 0.025)}.
[INFO 10-07 19:53:13] ax.service.ax_client: Generated new trial 52 with parameters {'x': 0.903903}.
[INFO 10-07 19:53:13] ax.service.ax_client: Completed trial 52 with data: {'y': (0.852357, 0.025)}.
[INFO 10-07 19:53:13] ax.service.ax_client: Generated new trial 53 with parameters {'x': 0.903231}.
[INFO 10-07 19:53:13] ax.service.ax_client: Completed trial 53 with data: {'y': (0.726902, 0.025)}.
[INFO 10-07 19:53:13] ax.service.ax_client: Generated new trial 54 with parameters {'x': 0.900749}.
[INFO 10-07 19:53:13] ax.service.ax_client: Completed trial 54 with data: {'y': (0.772191, 0.025)}.
[INFO 10-07 19:53:13] ax.service.ax_client: Generated new trial 55 with parameters {'x': 0.903554}.
[INFO 10-07 19:53:13] ax.service.ax_client: Completed trial 55 with data: {'y': (0.776993, 0.025)}.
[INFO 10-07 19:53:14] ax.service.ax_client: Generated new trial 56 with parameters {'x': 0.901248}.
[INFO 10-07 19:53:14] ax.service.ax_client: Completed trial 56 with data: {'y': (0.824798, 0.025)}.
[INFO 10-07 19:53:14] ax.service.ax_client: Generated new trial 57 with parameters {'x': 0.901014}.
[INFO 10-07 19:53:14] ax.service.ax_client: Completed trial 57 with data: {'y': (0.785034, 0.025)}.
[INFO 10-07 19:53:14] ax.service.ax_client: Generated new trial 58 with parameters {'x': 0.907044}.
[INFO 10-07 19:53:14] ax.service.ax_client: Completed trial 58 with data: {'y': (0.764855, 0.025)}.
[INFO 10-07 19:53:15] ax.service.ax_client: Generated new trial 59 with parameters {'x': 0.900126}.
[INFO 10-07 19:53:15] ax.service.ax_client: Completed trial 59 with data: {'y': (0.868459, 0.025)}.
[INFO 10-07 19:53:15] ax.service.ax_client: Generated new trial 60 with parameters {'x': 0.896666}.
[INFO 10-07 19:53:15] ax.service.ax_client: Completed trial 60 with data: {'y': (0.801584, 0.025)}.
[INFO 10-07 19:53:16] ax.service.ax_client: Generated new trial 61 with parameters {'x': 0.901012}.
[INFO 10-07 19:53:16] ax.service.ax_client: Completed trial 61 with data: {'y': (0.801098, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:16] ax.service.ax_client: Generated new trial 62 with parameters {'x': 0.898892}.
[INFO 10-07 19:53:16] ax.service.ax_client: Completed trial 62 with data: {'y': (0.806235, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:16] ax.service.ax_client: Generated new trial 63 with parameters {'x': 0.900486}.
[INFO 10-07 19:53:16] ax.service.ax_client: Completed trial 63 with data: {'y': (0.818956, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:17] ax.service.ax_client: Generated new trial 64 with parameters {'x': 0.899529}.
[INFO 10-07 19:53:17] ax.service.ax_client: Completed trial 64 with data: {'y': (0.841335, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:17] ax.service.ax_client: Generated new trial 65 with parameters {'x': 0.902763}.
[INFO 10-07 19:53:17] ax.service.ax_client: Completed trial 65 with data: {'y': (0.856247, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:17] ax.service.ax_client: Generated new trial 66 with parameters {'x': 0.901475}.
[INFO 10-07 19:53:17] ax.service.ax_client: Completed trial 66 with data: {'y': (0.874397, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:18] ax.service.ax_client: Generated new trial 67 with parameters {'x': 0.902133}.
[INFO 10-07 19:53:18] ax.service.ax_client: Completed trial 67 with data: {'y': (0.846737, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:18] ax.service.ax_client: Generated new trial 68 with parameters {'x': 0.901754}.
[INFO 10-07 19:53:18] ax.service.ax_client: Completed trial 68 with data: {'y': (0.824063, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:19] ax.service.ax_client: Generated new trial 69 with parameters {'x': 0.900354}.
[INFO 10-07 19:53:19] ax.service.ax_client: Completed trial 69 with data: {'y': (0.925403, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:19] ax.service.ax_client: Generated new trial 70 with parameters {'x': 0.873142}.
[INFO 10-07 19:53:19] ax.service.ax_client: Completed trial 70 with data: {'y': (0.411487, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:20] ax.service.ax_client: Generated new trial 71 with parameters {'x': 0.899928}.
[INFO 10-07 19:53:20] ax.service.ax_client: Completed trial 71 with data: {'y': (0.825494, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:20] ax.service.ax_client: Generated new trial 72 with parameters {'x': 0.901003}.
[INFO 10-07 19:53:20] ax.service.ax_client: Completed trial 72 with data: {'y': (0.863797, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:20] ax.service.ax_client: Generated new trial 73 with parameters {'x': 0.900778}.
[INFO 10-07 19:53:20] ax.service.ax_client: Completed trial 73 with data: {'y': (0.802959, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:21] ax.service.ax_client: Generated new trial 74 with parameters {'x': 0.699431}.
[INFO 10-07 19:53:21] ax.service.ax_client: Completed trial 74 with data: {'y': (0.536392, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:21] ax.service.ax_client: Generated new trial 75 with parameters {'x': 0.900211}.
[INFO 10-07 19:53:21] ax.service.ax_client: Completed trial 75 with data: {'y': (0.680727, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:22] ax.service.ax_client: Generated new trial 76 with parameters {'x': 0.898142}.
[INFO 10-07 19:53:22] ax.service.ax_client: Completed trial 76 with data: {'y': (0.796772, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:22] ax.service.ax_client: Generated new trial 77 with parameters {'x': 0.899835}.
[INFO 10-07 19:53:22] ax.service.ax_client: Completed trial 77 with data: {'y': (0.764079, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:23] ax.service.ax_client: Generated new trial 78 with parameters {'x': 0.902727}.
[INFO 10-07 19:53:23] ax.service.ax_client: Completed trial 78 with data: {'y': (0.816378, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:23] ax.service.ax_client: Generated new trial 79 with parameters {'x': 0.901757}.
[INFO 10-07 19:53:23] ax.service.ax_client: Completed trial 79 with data: {'y': (0.800264, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:24] ax.service.ax_client: Generated new trial 80 with parameters {'x': 0.899118}.
[INFO 10-07 19:53:24] ax.service.ax_client: Completed trial 80 with data: {'y': (0.819166, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:24] ax.service.ax_client: Generated new trial 81 with parameters {'x': 0.902129}.
[INFO 10-07 19:53:24] ax.service.ax_client: Completed trial 81 with data: {'y': (0.856783, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:25] ax.service.ax_client: Generated new trial 82 with parameters {'x': 0.901206}.
[INFO 10-07 19:53:25] ax.service.ax_client: Completed trial 82 with data: {'y': (0.760308, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:25] ax.service.ax_client: Generated new trial 83 with parameters {'x': 0.901346}.
[INFO 10-07 19:53:26] ax.service.ax_client: Completed trial 83 with data: {'y': (0.758607, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:26] ax.service.ax_client: Generated new trial 84 with parameters {'x': 0.898013}.
[INFO 10-07 19:53:26] ax.service.ax_client: Completed trial 84 with data: {'y': (0.853141, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:27] ax.service.ax_client: Generated new trial 85 with parameters {'x': 0.901941}.
[INFO 10-07 19:53:27] ax.service.ax_client: Completed trial 85 with data: {'y': (0.859983, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:27] ax.service.ax_client: Generated new trial 86 with parameters {'x': 0.901565}.
[INFO 10-07 19:53:27] ax.service.ax_client: Completed trial 86 with data: {'y': (0.752483, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:28] ax.service.ax_client: Generated new trial 87 with parameters {'x': 0.90233}.
[INFO 10-07 19:53:28] ax.service.ax_client: Completed trial 87 with data: {'y': (0.814213, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:29] ax.service.ax_client: Generated new trial 88 with parameters {'x': 0.896569}.
[INFO 10-07 19:53:29] ax.service.ax_client: Completed trial 88 with data: {'y': (0.719923, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:29] ax.service.ax_client: Generated new trial 89 with parameters {'x': 0.904786}.
[INFO 10-07 19:53:29] ax.service.ax_client: Completed trial 89 with data: {'y': (0.800252, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:30] ax.service.ax_client: Generated new trial 90 with parameters {'x': 0.901496}.
[INFO 10-07 19:53:30] ax.service.ax_client: Completed trial 90 with data: {'y': (0.782678, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:31] ax.service.ax_client: Generated new trial 91 with parameters {'x': 0.904171}.
[INFO 10-07 19:53:31] ax.service.ax_client: Completed trial 91 with data: {'y': (0.792567, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:31] ax.service.ax_client: Generated new trial 92 with parameters {'x': 0.90164}.
[INFO 10-07 19:53:31] ax.service.ax_client: Completed trial 92 with data: {'y': (0.853484, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:32] ax.service.ax_client: Generated new trial 93 with parameters {'x': 0.901843}.
[INFO 10-07 19:53:32] ax.service.ax_client: Completed trial 93 with data: {'y': (0.800771, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:33] ax.service.ax_client: Generated new trial 94 with parameters {'x': 0.902591}.
[INFO 10-07 19:53:33] ax.service.ax_client: Completed trial 94 with data: {'y': (0.763648, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:33] ax.service.ax_client: Generated new trial 95 with parameters {'x': 0.900607}.
[INFO 10-07 19:53:33] ax.service.ax_client: Completed trial 95 with data: {'y': (0.784908, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:34] ax.service.ax_client: Generated new trial 96 with parameters {'x': 0.901091}.
[INFO 10-07 19:53:34] ax.service.ax_client: Completed trial 96 with data: {'y': (0.766167, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:35] ax.service.ax_client: Generated new trial 97 with parameters {'x': 0.900868}.
[INFO 10-07 19:53:35] ax.service.ax_client: Completed trial 97 with data: {'y': (0.839752, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:36] ax.service.ax_client: Generated new trial 98 with parameters {'x': 0.902254}.
[INFO 10-07 19:53:36] ax.service.ax_client: Completed trial 98 with data: {'y': (0.776735, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:53:37] ax.service.ax_client: Generated new trial 99 with parameters {'x': 0.897836}.
[INFO 10-07 19:53:37] ax.service.ax_client: Completed trial 99 with data: {'y': (0.75079, 0.025)}.
[INFO 10-07 19:53:37] ax.service.ax_client: Starting optimization with verbose logging. To disable logging, set the `verbose_logging` argument to `False`. Note that float values in the logs are rounded to 6 decimal points.
[INFO 10-07 19:53:37] ax.service.utils.instantiation: Inferred value type of ParameterType.FLOAT for parameter x. If that is not the expected value type, you can explicity specify 'value_type' ('int', 'float', 'bool' or 'str') in parameter dict.
[INFO 10-07 19:53:37] ax.service.utils.instantiation: Created search space: SearchSpace(parameters=[RangeParameter(name='x', parameter_type=FLOAT, range=[0.0, 1.0])], parameter_constraints=[]).
[INFO 10-07 19:53:37] ax.modelbridge.dispatch_utils: Using Bayesian optimization since there are more ordered parameters than there are categories for the unordered categorical parameters.
[INFO 10-07 19:53:37] ax.modelbridge.dispatch_utils: Using Bayesian Optimization generation strategy: GenerationStrategy(name='Sobol+GPEI', steps=[Sobol for 5 trials, GPEI for subsequent trials]). Iterations after 5 will take longer to generate due to model-fitting.
[INFO 10-07 19:53:37] ax.service.ax_client: Generated new trial 0 with parameters {'x': 0.542335}.
[INFO 10-07 19:53:37] ax.service.ax_client: Completed trial 0 with data: {'y': (0.019285, 0.025)}.
[INFO 10-07 19:53:37] ax.service.ax_client: Generated new trial 1 with parameters {'x': 0.615157}.
[INFO 10-07 19:53:37] ax.service.ax_client: Completed trial 1 with data: {'y': (0.066619, 0.025)}.
[INFO 10-07 19:53:37] ax.service.ax_client: Generated new trial 2 with parameters {'x': 0.439061}.
[INFO 10-07 19:53:37] ax.service.ax_client: Completed trial 2 with data: {'y': (-0.002942, 0.025)}.
[INFO 10-07 19:53:37] ax.service.ax_client: Generated new trial 3 with parameters {'x': 0.422258}.
[INFO 10-07 19:53:37] ax.service.ax_client: Completed trial 3 with data: {'y': (-0.091197, 0.025)}.
[INFO 10-07 19:53:37] ax.service.ax_client: Generated new trial 4 with parameters {'x': 0.951257}.
[INFO 10-07 19:53:37] ax.service.ax_client: Completed trial 4 with data: {'y': (0.011238, 0.025)}.
[INFO 10-07 19:53:37] ax.service.ax_client: Generated new trial 5 with parameters {'x': 0.728356}.
[INFO 10-07 19:53:37] ax.service.ax_client: Completed trial 5 with data: {'y': (0.316955, 0.025)}.
[INFO 10-07 19:53:37] ax.service.ax_client: Generated new trial 6 with parameters {'x': 0.781105}.
[INFO 10-07 19:53:37] ax.service.ax_client: Completed trial 6 with data: {'y': (-0.092643, 0.025)}.
[INFO 10-07 19:53:38] ax.service.ax_client: Generated new trial 7 with parameters {'x': 0.704542}.
[INFO 10-07 19:53:38] ax.service.ax_client: Completed trial 7 with data: {'y': (0.573242, 0.025)}.
[INFO 10-07 19:53:38] ax.service.ax_client: Generated new trial 8 with parameters {'x': 0.687295}.
[INFO 10-07 19:53:38] ax.service.ax_client: Completed trial 8 with data: {'y': (0.463095, 0.025)}.
[INFO 10-07 19:53:38] ax.service.ax_client: Generated new trial 9 with parameters {'x': 0.700692}.
[INFO 10-07 19:53:38] ax.service.ax_client: Completed trial 9 with data: {'y': (0.486821, 0.025)}.
[INFO 10-07 19:53:39] ax.service.ax_client: Generated new trial 10 with parameters {'x': 0.0}.
[INFO 10-07 19:53:39] ax.service.ax_client: Completed trial 10 with data: {'y': (-0.026726, 0.025)}.
[INFO 10-07 19:53:39] ax.service.ax_client: Generated new trial 11 with parameters {'x': 0.193103}.
[INFO 10-07 19:53:39] ax.service.ax_client: Completed trial 11 with data: {'y': (-0.006278, 0.025)}.
[INFO 10-07 19:53:39] ax.service.ax_client: Generated new trial 12 with parameters {'x': 0.292527}.
[INFO 10-07 19:53:39] ax.service.ax_client: Completed trial 12 with data: {'y': (0.032322, 0.025)}.
[INFO 10-07 19:53:39] ax.service.ax_client: Generated new trial 13 with parameters {'x': 0.707491}.
[INFO 10-07 19:53:39] ax.service.ax_client: Completed trial 13 with data: {'y': (0.533321, 0.025)}.
[INFO 10-07 19:53:40] ax.service.ax_client: Generated new trial 14 with parameters {'x': 0.702655}.
[INFO 10-07 19:53:40] ax.service.ax_client: Completed trial 14 with data: {'y': (0.49155, 0.025)}.
[INFO 10-07 19:53:40] ax.service.ax_client: Generated new trial 15 with parameters {'x': 0.098392}.
[INFO 10-07 19:53:40] ax.service.ax_client: Completed trial 15 with data: {'y': (0.026103, 0.025)}.
[INFO 10-07 19:53:40] ax.service.ax_client: Generated new trial 16 with parameters {'x': 0.874007}.
[INFO 10-07 19:53:40] ax.service.ax_client: Completed trial 16 with data: {'y': (0.428477, 0.025)}.
[INFO 10-07 19:53:40] ax.service.ax_client: Generated new trial 17 with parameters {'x': 0.850726}.
[INFO 10-07 19:53:40] ax.service.ax_client: Completed trial 17 with data: {'y': (0.168513, 0.025)}.
[INFO 10-07 19:53:40] ax.service.ax_client: Generated new trial 18 with parameters {'x': 0.89458}.
[INFO 10-07 19:53:40] ax.service.ax_client: Completed trial 18 with data: {'y': (0.796961, 0.025)}.
[INFO 10-07 19:53:41] ax.service.ax_client: Generated new trial 19 with parameters {'x': 0.902246}.
[INFO 10-07 19:53:41] ax.service.ax_client: Completed trial 19 with data: {'y': (0.766262, 0.025)}.
[INFO 10-07 19:53:41] ax.service.ax_client: Generated new trial 20 with parameters {'x': 0.897738}.
[INFO 10-07 19:53:41] ax.service.ax_client: Completed trial 20 with data: {'y': (0.74495, 0.025)}.
[INFO 10-07 19:53:42] ax.service.ax_client: Generated new trial 21 with parameters {'x': 0.910454}.
[INFO 10-07 19:53:42] ax.service.ax_client: Completed trial 21 with data: {'y': (0.763353, 0.025)}.
[INFO 10-07 19:53:42] ax.service.ax_client: Generated new trial 22 with parameters {'x': 0.350964}.
[INFO 10-07 19:53:42] ax.service.ax_client: Completed trial 22 with data: {'y': (-0.025087, 0.025)}.
[INFO 10-07 19:53:42] ax.service.ax_client: Generated new trial 23 with parameters {'x': 0.905056}.
[INFO 10-07 19:53:42] ax.service.ax_client: Completed trial 23 with data: {'y': (0.854472, 0.025)}.
[INFO 10-07 19:53:42] ax.service.ax_client: Generated new trial 24 with parameters {'x': 0.903662}.
[INFO 10-07 19:53:42] ax.service.ax_client: Completed trial 24 with data: {'y': (0.760469, 0.025)}.
[INFO 10-07 19:53:43] ax.service.ax_client: Generated new trial 25 with parameters {'x': 0.491487}.
[INFO 10-07 19:53:43] ax.service.ax_client: Completed trial 25 with data: {'y': (0.253508, 0.025)}.
[INFO 10-07 19:53:43] ax.service.ax_client: Generated new trial 26 with parameters {'x': 0.243464}.
[INFO 10-07 19:53:43] ax.service.ax_client: Completed trial 26 with data: {'y': (-0.000261, 0.025)}.
[INFO 10-07 19:53:43] ax.service.ax_client: Generated new trial 27 with parameters {'x': 0.907142}.
[INFO 10-07 19:53:43] ax.service.ax_client: Completed trial 27 with data: {'y': (0.751981, 0.025)}.
[INFO 10-07 19:53:43] ax.service.ax_client: Generated new trial 28 with parameters {'x': 0.90038}.
[INFO 10-07 19:53:43] ax.service.ax_client: Completed trial 28 with data: {'y': (0.813721, 0.025)}.
[INFO 10-07 19:53:43] ax.service.ax_client: Generated new trial 29 with parameters {'x': 0.899198}.
[INFO 10-07 19:53:43] ax.service.ax_client: Completed trial 29 with data: {'y': (0.812934, 0.025)}.
[INFO 10-07 19:53:44] ax.service.ax_client: Generated new trial 30 with parameters {'x': 0.901369}.
[INFO 10-07 19:53:44] ax.service.ax_client: Completed trial 30 with data: {'y': (0.783377, 0.025)}.
[INFO 10-07 19:53:44] ax.service.ax_client: Generated new trial 31 with parameters {'x': 0.902938}.
[INFO 10-07 19:53:44] ax.service.ax_client: Completed trial 31 with data: {'y': (0.743824, 0.025)}.
[INFO 10-07 19:53:44] ax.service.ax_client: Generated new trial 32 with parameters {'x': 0.896025}.
[INFO 10-07 19:53:44] ax.service.ax_client: Completed trial 32 with data: {'y': (0.859923, 0.025)}.
[INFO 10-07 19:53:44] ax.service.ax_client: Generated new trial 33 with parameters {'x': 0.918172}.
[INFO 10-07 19:53:44] ax.service.ax_client: Completed trial 33 with data: {'y': (0.561865, 0.025)}.
[INFO 10-07 19:53:45] ax.service.ax_client: Generated new trial 34 with parameters {'x': 0.656059}.
[INFO 10-07 19:53:45] ax.service.ax_client: Completed trial 34 with data: {'y': (0.130627, 0.025)}.
[INFO 10-07 19:53:45] ax.service.ax_client: Generated new trial 35 with parameters {'x': 0.892642}.
[INFO 10-07 19:53:45] ax.service.ax_client: Completed trial 35 with data: {'y': (0.846788, 0.025)}.
[INFO 10-07 19:53:45] ax.service.ax_client: Generated new trial 36 with parameters {'x': 0.890617}.
[INFO 10-07 19:53:45] ax.service.ax_client: Completed trial 36 with data: {'y': (0.789571, 0.025)}.
[INFO 10-07 19:53:45] ax.service.ax_client: Generated new trial 37 with parameters {'x': 1.0}.
[INFO 10-07 19:53:45] ax.service.ax_client: Completed trial 37 with data: {'y': (0.036862, 0.025)}.
[INFO 10-07 19:53:46] ax.service.ax_client: Generated new trial 38 with parameters {'x': 0.893661}.
[INFO 10-07 19:53:46] ax.service.ax_client: Completed trial 38 with data: {'y': (0.831402, 0.025)}.
[INFO 10-07 19:53:46] ax.service.ax_client: Generated new trial 39 with parameters {'x': 0.895294}.
[INFO 10-07 19:53:46] ax.service.ax_client: Completed trial 39 with data: {'y': (0.812138, 0.025)}.
[INFO 10-07 19:53:46] ax.service.ax_client: Generated new trial 40 with parameters {'x': 0.050553}.
[INFO 10-07 19:53:46] ax.service.ax_client: Completed trial 40 with data: {'y': (-0.010268, 0.025)}.
[INFO 10-07 19:53:46] ax.service.ax_client: Generated new trial 41 with parameters {'x': 0.891854}.
[INFO 10-07 19:53:46] ax.service.ax_client: Completed trial 41 with data: {'y': (0.723625, 0.025)}.
[INFO 10-07 19:53:47] ax.service.ax_client: Generated new trial 42 with parameters {'x': 0.896901}.
[INFO 10-07 19:53:47] ax.service.ax_client: Completed trial 42 with data: {'y': (0.831568, 0.025)}.
[INFO 10-07 19:53:47] ax.service.ax_client: Generated new trial 43 with parameters {'x': 0.89645}.
[INFO 10-07 19:53:47] ax.service.ax_client: Completed trial 43 with data: {'y': (0.726796, 0.025)}.
[INFO 10-07 19:53:47] ax.service.ax_client: Generated new trial 44 with parameters {'x': 0.898372}.
[INFO 10-07 19:53:47] ax.service.ax_client: Completed trial 44 with data: {'y': (0.925368, 0.025)}.
[INFO 10-07 19:53:48] ax.service.ax_client: Generated new trial 45 with parameters {'x': 0.89732}.
[INFO 10-07 19:53:48] ax.service.ax_client: Completed trial 45 with data: {'y': (0.761866, 0.025)}.
[INFO 10-07 19:53:48] ax.service.ax_client: Generated new trial 46 with parameters {'x': 0.8997}.
[INFO 10-07 19:53:48] ax.service.ax_client: Completed trial 46 with data: {'y': (0.850022, 0.025)}.
[INFO 10-07 19:53:48] ax.service.ax_client: Generated new trial 47 with parameters {'x': 0.898839}.
[INFO 10-07 19:53:48] ax.service.ax_client: Completed trial 47 with data: {'y': (0.872887, 0.025)}.
[INFO 10-07 19:53:48] ax.service.ax_client: Generated new trial 48 with parameters {'x': 0.898052}.
[INFO 10-07 19:53:48] ax.service.ax_client: Completed trial 48 with data: {'y': (0.829411, 0.025)}.
[INFO 10-07 19:53:49] ax.service.ax_client: Generated new trial 49 with parameters {'x': 0.897536}.
[INFO 10-07 19:53:49] ax.service.ax_client: Completed trial 49 with data: {'y': (0.782991, 0.025)}.
[INFO 10-07 19:53:49] ax.service.ax_client: Generated new trial 50 with parameters {'x': 0.898586}.
[INFO 10-07 19:53:49] ax.service.ax_client: Completed trial 50 with data: {'y': (0.805214, 0.025)}.
[INFO 10-07 19:53:49] ax.service.ax_client: Generated new trial 51 with parameters {'x': 0.900872}.
[INFO 10-07 19:53:49] ax.service.ax_client: Completed trial 51 with data: {'y': (0.87835, 0.025)}.
[INFO 10-07 19:53:50] ax.service.ax_client: Generated new trial 52 with parameters {'x': 0.900048}.
[INFO 10-07 19:53:50] ax.service.ax_client: Completed trial 52 with data: {'y': (0.817461, 0.025)}.
[INFO 10-07 19:53:50] ax.service.ax_client: Generated new trial 53 with parameters {'x': 0.899425}.
[INFO 10-07 19:53:50] ax.service.ax_client: Completed trial 53 with data: {'y': (0.822554, 0.025)}.
[INFO 10-07 19:53:50] ax.service.ax_client: Generated new trial 54 with parameters {'x': 0.89987}.
[INFO 10-07 19:53:50] ax.service.ax_client: Completed trial 54 with data: {'y': (0.868119, 0.025)}.
[INFO 10-07 19:53:50] ax.service.ax_client: Generated new trial 55 with parameters {'x': 0.898973}.
[INFO 10-07 19:53:50] ax.service.ax_client: Completed trial 55 with data: {'y': (0.812715, 0.025)}.
[INFO 10-07 19:53:51] ax.service.ax_client: Generated new trial 56 with parameters {'x': 0.895692}.
[INFO 10-07 19:53:51] ax.service.ax_client: Completed trial 56 with data: {'y': (0.845764, 0.025)}.
[INFO 10-07 19:53:51] ax.service.ax_client: Generated new trial 57 with parameters {'x': 0.897087}.
[INFO 10-07 19:53:51] ax.service.ax_client: Completed trial 57 with data: {'y': (0.716958, 0.025)}.
[INFO 10-07 19:53:51] ax.service.ax_client: Generated new trial 58 with parameters {'x': 0.900591}.
[INFO 10-07 19:53:51] ax.service.ax_client: Completed trial 58 with data: {'y': (0.941549, 0.025)}.
[INFO 10-07 19:53:52] ax.service.ax_client: Generated new trial 59 with parameters {'x': 0.885592}.
[INFO 10-07 19:53:52] ax.service.ax_client: Completed trial 59 with data: {'y': (0.668848, 0.025)}.
[INFO 10-07 19:53:52] ax.service.ax_client: Generated new trial 60 with parameters {'x': 0.9136}.
[INFO 10-07 19:53:52] ax.service.ax_client: Completed trial 60 with data: {'y': (0.74732, 0.025)}.
[INFO 10-07 19:53:53] ax.service.ax_client: Generated new trial 61 with parameters {'x': 0.67727}.
[INFO 10-07 19:53:53] ax.service.ax_client: Completed trial 61 with data: {'y': (0.294309, 0.025)}.
[INFO 10-07 19:53:53] ax.service.ax_client: Generated new trial 62 with parameters {'x': 0.742055}.
[INFO 10-07 19:53:53] ax.service.ax_client: Completed trial 62 with data: {'y': (0.077509, 0.025)}.
[INFO 10-07 19:53:54] ax.service.ax_client: Generated new trial 63 with parameters {'x': 0.717776}.
[INFO 10-07 19:53:54] ax.service.ax_client: Completed trial 63 with data: {'y': (0.356942, 0.025)}.
[INFO 10-07 19:53:54] ax.service.ax_client: Generated new trial 64 with parameters {'x': 0.90168}.
[INFO 10-07 19:53:54] ax.service.ax_client: Completed trial 64 with data: {'y': (0.788754, 0.025)}.
[INFO 10-07 19:53:54] ax.service.ax_client: Generated new trial 65 with parameters {'x': 0.902553}.
[INFO 10-07 19:53:54] ax.service.ax_client: Completed trial 65 with data: {'y': (0.815374, 0.025)}.
[INFO 10-07 19:53:55] ax.service.ax_client: Generated new trial 66 with parameters {'x': 0.901121}.
[INFO 10-07 19:53:55] ax.service.ax_client: Completed trial 66 with data: {'y': (0.825749, 0.025)}.
[INFO 10-07 19:53:55] ax.service.ax_client: Generated new trial 67 with parameters {'x': 0.900195}.
[INFO 10-07 19:53:55] ax.service.ax_client: Completed trial 67 with data: {'y': (0.797014, 0.025)}.
[INFO 10-07 19:53:55] ax.service.ax_client: Generated new trial 68 with parameters {'x': 0.898219}.
[INFO 10-07 19:53:55] ax.service.ax_client: Completed trial 68 with data: {'y': (0.831863, 0.025)}.
[INFO 10-07 19:53:56] ax.service.ax_client: Generated new trial 69 with parameters {'x': 0.89872}.
[INFO 10-07 19:53:56] ax.service.ax_client: Completed trial 69 with data: {'y': (0.879538, 0.025)}.
[INFO 10-07 19:53:56] ax.service.ax_client: Generated new trial 70 with parameters {'x': 0.897959}.
[INFO 10-07 19:53:56] ax.service.ax_client: Completed trial 70 with data: {'y': (0.815808, 0.025)}.
[INFO 10-07 19:53:57] ax.service.ax_client: Generated new trial 71 with parameters {'x': 0.899562}.
[INFO 10-07 19:53:57] ax.service.ax_client: Completed trial 71 with data: {'y': (0.822751, 0.025)}.
[INFO 10-07 19:53:57] ax.service.ax_client: Generated new trial 72 with parameters {'x': 0.901943}.
[INFO 10-07 19:53:57] ax.service.ax_client: Completed trial 72 with data: {'y': (0.709263, 0.025)}.
[INFO 10-07 19:53:58] ax.service.ax_client: Generated new trial 73 with parameters {'x': 0.474361}.
[INFO 10-07 19:53:58] ax.service.ax_client: Completed trial 73 with data: {'y': (0.121489, 0.025)}.
[INFO 10-07 19:53:58] ax.service.ax_client: Generated new trial 74 with parameters {'x': 0.898479}.
[INFO 10-07 19:53:58] ax.service.ax_client: Completed trial 74 with data: {'y': (0.821309, 0.025)}.
[INFO 10-07 19:53:58] ax.service.ax_client: Generated new trial 75 with parameters {'x': 0.899287}.
[INFO 10-07 19:53:58] ax.service.ax_client: Completed trial 75 with data: {'y': (0.799837, 0.025)}.
[INFO 10-07 19:53:59] ax.service.ax_client: Generated new trial 76 with parameters {'x': 0.896702}.
[INFO 10-07 19:53:59] ax.service.ax_client: Completed trial 76 with data: {'y': (0.744797, 0.025)}.
[INFO 10-07 19:53:59] ax.service.ax_client: Generated new trial 77 with parameters {'x': 0.880489}.
[INFO 10-07 19:53:59] ax.service.ax_client: Completed trial 77 with data: {'y': (0.583999, 0.025)}.
[INFO 10-07 19:54:00] ax.service.ax_client: Generated new trial 78 with parameters {'x': 0.899089}.
[INFO 10-07 19:54:00] ax.service.ax_client: Completed trial 78 with data: {'y': (0.775514, 0.025)}.
[INFO 10-07 19:54:00] ax.service.ax_client: Generated new trial 79 with parameters {'x': 0.895075}.
[INFO 10-07 19:54:00] ax.service.ax_client: Completed trial 79 with data: {'y': (0.75071, 0.025)}.
[INFO 10-07 19:54:01] ax.service.ax_client: Generated new trial 80 with parameters {'x': 0.900297}.
[INFO 10-07 19:54:01] ax.service.ax_client: Completed trial 80 with data: {'y': (0.798118, 0.025)}.
[INFO 10-07 19:54:01] ax.service.ax_client: Generated new trial 81 with parameters {'x': 0.899639}.
[INFO 10-07 19:54:01] ax.service.ax_client: Completed trial 81 with data: {'y': (0.819868, 0.025)}.
[INFO 10-07 19:54:02] ax.service.ax_client: Generated new trial 82 with parameters {'x': 0.904585}.
[INFO 10-07 19:54:02] ax.service.ax_client: Completed trial 82 with data: {'y': (0.809242, 0.025)}.
[INFO 10-07 19:54:02] ax.service.ax_client: Generated new trial 83 with parameters {'x': 0.899199}.
[INFO 10-07 19:54:02] ax.service.ax_client: Completed trial 83 with data: {'y': (0.79357, 0.025)}.
[INFO 10-07 19:54:03] ax.service.ax_client: Generated new trial 84 with parameters {'x': 0.899781}.
[INFO 10-07 19:54:03] ax.service.ax_client: Completed trial 84 with data: {'y': (0.80993, 0.025)}.
/opt/conda/lib/python3.9/site-packages/botorch/optim/optimize.py:306: RuntimeWarning:
Optimization failed in `gen_candidates_scipy` with the following warning(s):
[NumericalWarning('A not p.d., added jitter of 1.0e-08 to the diagonal'), OptimizationWarning('Optimization failed within `scipy.optimize.minimize` with status 2.'), NumericalWarning('A not p.d., added jitter of 1.0e-08 to the diagonal')]
Trying again with a new set of initial conditions.
[INFO 10-07 19:54:04] ax.service.ax_client: Generated new trial 85 with parameters {'x': 0.898588}.
[INFO 10-07 19:54:04] ax.service.ax_client: Completed trial 85 with data: {'y': (0.800104, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:04] ax.service.ax_client: Generated new trial 86 with parameters {'x': 0.900876}.
[INFO 10-07 19:54:04] ax.service.ax_client: Completed trial 86 with data: {'y': (0.835769, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:05] ax.service.ax_client: Generated new trial 87 with parameters {'x': 0.900121}.
[INFO 10-07 19:54:05] ax.service.ax_client: Completed trial 87 with data: {'y': (0.790893, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:05] ax.service.ax_client: Generated new trial 88 with parameters {'x': 0.895123}.
[INFO 10-07 19:54:05] ax.service.ax_client: Completed trial 88 with data: {'y': (0.830161, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:06] ax.service.ax_client: Generated new trial 89 with parameters {'x': 0.894945}.
[INFO 10-07 19:54:06] ax.service.ax_client: Completed trial 89 with data: {'y': (0.736699, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:06] ax.service.ax_client: Generated new trial 90 with parameters {'x': 0.90067}.
[INFO 10-07 19:54:06] ax.service.ax_client: Completed trial 90 with data: {'y': (0.80689, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:07] ax.service.ax_client: Generated new trial 91 with parameters {'x': 0.899018}.
[INFO 10-07 19:54:07] ax.service.ax_client: Completed trial 91 with data: {'y': (0.779061, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:07] ax.service.ax_client: Generated new trial 92 with parameters {'x': 0.901651}.
[INFO 10-07 19:54:07] ax.service.ax_client: Completed trial 92 with data: {'y': (0.789238, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:08] ax.service.ax_client: Generated new trial 93 with parameters {'x': 0.900554}.
[INFO 10-07 19:54:08] ax.service.ax_client: Completed trial 93 with data: {'y': (0.78325, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:08] ax.service.ax_client: Generated new trial 94 with parameters {'x': 0.901147}.
[INFO 10-07 19:54:08] ax.service.ax_client: Completed trial 94 with data: {'y': (0.836587, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:09] ax.service.ax_client: Generated new trial 95 with parameters {'x': 0.901144}.
[INFO 10-07 19:54:09] ax.service.ax_client: Completed trial 95 with data: {'y': (0.871852, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:10] ax.service.ax_client: Generated new trial 96 with parameters {'x': 0.895733}.
[INFO 10-07 19:54:10] ax.service.ax_client: Completed trial 96 with data: {'y': (0.716225, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:11] ax.service.ax_client: Generated new trial 97 with parameters {'x': 0.902851}.
[INFO 10-07 19:54:11] ax.service.ax_client: Completed trial 97 with data: {'y': (0.805198, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:11] ax.service.ax_client: Generated new trial 98 with parameters {'x': 0.895984}.
[INFO 10-07 19:54:11] ax.service.ax_client: Completed trial 98 with data: {'y': (0.778182, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:12] ax.service.ax_client: Generated new trial 99 with parameters {'x': 0.901448}.
[INFO 10-07 19:54:12] ax.service.ax_client: Completed trial 99 with data: {'y': (0.896395, 0.025)}.
[INFO 10-07 19:54:12] ax.service.ax_client: Starting optimization with verbose logging. To disable logging, set the `verbose_logging` argument to `False`. Note that float values in the logs are rounded to 6 decimal points.
[INFO 10-07 19:54:12] ax.service.utils.instantiation: Inferred value type of ParameterType.FLOAT for parameter x. If that is not the expected value type, you can explicity specify 'value_type' ('int', 'float', 'bool' or 'str') in parameter dict.
[INFO 10-07 19:54:12] ax.service.utils.instantiation: Created search space: SearchSpace(parameters=[RangeParameter(name='x', parameter_type=FLOAT, range=[0.0, 1.0])], parameter_constraints=[]).
[INFO 10-07 19:54:12] ax.modelbridge.dispatch_utils: Using Bayesian optimization since there are more ordered parameters than there are categories for the unordered categorical parameters.
[INFO 10-07 19:54:12] ax.modelbridge.dispatch_utils: Using Bayesian Optimization generation strategy: GenerationStrategy(name='Sobol+GPEI', steps=[Sobol for 5 trials, GPEI for subsequent trials]). Iterations after 5 will take longer to generate due to model-fitting.
[INFO 10-07 19:54:12] ax.service.ax_client: Generated new trial 0 with parameters {'x': 0.349964}.
[INFO 10-07 19:54:12] ax.service.ax_client: Completed trial 0 with data: {'y': (0.105569, 0.025)}.
[INFO 10-07 19:54:12] ax.service.ax_client: Generated new trial 1 with parameters {'x': 0.666901}.
[INFO 10-07 19:54:12] ax.service.ax_client: Completed trial 1 with data: {'y': (0.207742, 0.025)}.
[INFO 10-07 19:54:12] ax.service.ax_client: Generated new trial 2 with parameters {'x': 0.732177}.
[INFO 10-07 19:54:12] ax.service.ax_client: Completed trial 2 with data: {'y': (0.191451, 0.025)}.
[INFO 10-07 19:54:12] ax.service.ax_client: Generated new trial 3 with parameters {'x': 0.607162}.
[INFO 10-07 19:54:12] ax.service.ax_client: Completed trial 3 with data: {'y': (0.066768, 0.025)}.
[INFO 10-07 19:54:12] ax.service.ax_client: Generated new trial 4 with parameters {'x': 0.011369}.
[INFO 10-07 19:54:12] ax.service.ax_client: Completed trial 4 with data: {'y': (0.043598, 0.025)}.
[INFO 10-07 19:54:12] ax.service.ax_client: Generated new trial 5 with parameters {'x': 0.878902}.
[INFO 10-07 19:54:12] ax.service.ax_client: Completed trial 5 with data: {'y': (0.558123, 0.025)}.
[INFO 10-07 19:54:12] ax.service.ax_client: Generated new trial 6 with parameters {'x': 0.95278}.
[INFO 10-07 19:54:12] ax.service.ax_client: Completed trial 6 with data: {'y': (0.140117, 0.025)}.
[INFO 10-07 19:54:13] ax.service.ax_client: Generated new trial 7 with parameters {'x': 0.850498}.
[INFO 10-07 19:54:13] ax.service.ax_client: Completed trial 7 with data: {'y': (0.148019, 0.025)}.
[INFO 10-07 19:54:13] ax.service.ax_client: Generated new trial 8 with parameters {'x': 0.889297}.
[INFO 10-07 19:54:13] ax.service.ax_client: Completed trial 8 with data: {'y': (0.72814, 0.025)}.
[INFO 10-07 19:54:13] ax.service.ax_client: Generated new trial 9 with parameters {'x': 0.899501}.
[INFO 10-07 19:54:13] ax.service.ax_client: Completed trial 9 with data: {'y': (0.720998, 0.025)}.
[INFO 10-07 19:54:14] ax.service.ax_client: Generated new trial 10 with parameters {'x': 0.894458}.
[INFO 10-07 19:54:14] ax.service.ax_client: Completed trial 10 with data: {'y': (0.678657, 0.025)}.
[INFO 10-07 19:54:14] ax.service.ax_client: Generated new trial 11 with parameters {'x': 0.907169}.
[INFO 10-07 19:54:14] ax.service.ax_client: Completed trial 11 with data: {'y': (0.811925, 0.025)}.
[INFO 10-07 19:54:15] ax.service.ax_client: Generated new trial 12 with parameters {'x': 0.914923}.
[INFO 10-07 19:54:15] ax.service.ax_client: Completed trial 12 with data: {'y': (0.77172, 0.025)}.
[INFO 10-07 19:54:15] ax.service.ax_client: Generated new trial 13 with parameters {'x': 0.910344}.
[INFO 10-07 19:54:15] ax.service.ax_client: Completed trial 13 with data: {'y': (0.759666, 0.025)}.
[INFO 10-07 19:54:15] ax.service.ax_client: Generated new trial 14 with parameters {'x': 0.188241}.
[INFO 10-07 19:54:15] ax.service.ax_client: Completed trial 14 with data: {'y': (0.011744, 0.025)}.
[INFO 10-07 19:54:15] ax.service.ax_client: Generated new trial 15 with parameters {'x': 0.471375}.
[INFO 10-07 19:54:15] ax.service.ax_client: Completed trial 15 with data: {'y': (0.148745, 0.025)}.
[INFO 10-07 19:54:16] ax.service.ax_client: Generated new trial 16 with parameters {'x': 0.098097}.
[INFO 10-07 19:54:16] ax.service.ax_client: Completed trial 16 with data: {'y': (0.008056, 0.025)}.
[INFO 10-07 19:54:16] ax.service.ax_client: Generated new trial 17 with parameters {'x': 0.274016}.
[INFO 10-07 19:54:16] ax.service.ax_client: Completed trial 17 with data: {'y': (0.016991, 0.025)}.
[INFO 10-07 19:54:16] ax.service.ax_client: Generated new trial 18 with parameters {'x': 0.912354}.
[INFO 10-07 19:54:16] ax.service.ax_client: Completed trial 18 with data: {'y': (0.73752, 0.025)}.
[INFO 10-07 19:54:16] ax.service.ax_client: Generated new trial 19 with parameters {'x': 0.535187}.
[INFO 10-07 19:54:16] ax.service.ax_client: Completed trial 19 with data: {'y': (0.112241, 0.025)}.
[INFO 10-07 19:54:16] ax.service.ax_client: Generated new trial 20 with parameters {'x': 0.904996}.
[INFO 10-07 19:54:16] ax.service.ax_client: Completed trial 20 with data: {'y': (0.791157, 0.025)}.
[INFO 10-07 19:54:17] ax.service.ax_client: Generated new trial 21 with parameters {'x': 0.411873}.
[INFO 10-07 19:54:17] ax.service.ax_client: Completed trial 21 with data: {'y': (-0.057685, 0.025)}.
[INFO 10-07 19:54:17] ax.service.ax_client: Generated new trial 22 with parameters {'x': 0.908635}.
[INFO 10-07 19:54:17] ax.service.ax_client: Completed trial 22 with data: {'y': (0.885805, 0.025)}.
[INFO 10-07 19:54:17] ax.service.ax_client: Generated new trial 23 with parameters {'x': 0.921933}.
[INFO 10-07 19:54:17] ax.service.ax_client: Completed trial 23 with data: {'y': (0.636867, 0.025)}.
[INFO 10-07 19:54:17] ax.service.ax_client: Generated new trial 24 with parameters {'x': 0.907924}.
[INFO 10-07 19:54:17] ax.service.ax_client: Completed trial 24 with data: {'y': (0.797661, 0.025)}.
[INFO 10-07 19:54:17] ax.service.ax_client: Generated new trial 25 with parameters {'x': 0.78349}.
[INFO 10-07 19:54:17] ax.service.ax_client: Completed trial 25 with data: {'y': (-0.051702, 0.025)}.
[INFO 10-07 19:54:17] ax.service.ax_client: Generated new trial 26 with parameters {'x': 0.906287}.
[INFO 10-07 19:54:18] ax.service.ax_client: Completed trial 26 with data: {'y': (0.875263, 0.025)}.
[INFO 10-07 19:54:18] ax.service.ax_client: Generated new trial 27 with parameters {'x': 1.0}.
[INFO 10-07 19:54:18] ax.service.ax_client: Completed trial 27 with data: {'y': (0.042028, 0.025)}.
[INFO 10-07 19:54:18] ax.service.ax_client: Generated new trial 28 with parameters {'x': 0.906796}.
[INFO 10-07 19:54:18] ax.service.ax_client: Completed trial 28 with data: {'y': (0.849308, 0.025)}.
[INFO 10-07 19:54:18] ax.service.ax_client: Generated new trial 29 with parameters {'x': 0.905704}.
[INFO 10-07 19:54:18] ax.service.ax_client: Completed trial 29 with data: {'y': (0.792969, 0.025)}.
[INFO 10-07 19:54:18] ax.service.ax_client: Generated new trial 30 with parameters {'x': 0.90916}.
[INFO 10-07 19:54:18] ax.service.ax_client: Completed trial 30 with data: {'y': (0.66472, 0.025)}.
[INFO 10-07 19:54:19] ax.service.ax_client: Generated new trial 31 with parameters {'x': 0.90402}.
[INFO 10-07 19:54:19] ax.service.ax_client: Completed trial 31 with data: {'y': (0.908199, 0.025)}.
[INFO 10-07 19:54:19] ax.service.ax_client: Generated new trial 32 with parameters {'x': 0.698518}.
[INFO 10-07 19:54:19] ax.service.ax_client: Completed trial 32 with data: {'y': (0.48115, 0.025)}.
[INFO 10-07 19:54:19] ax.service.ax_client: Generated new trial 33 with parameters {'x': 0.711768}.
[INFO 10-07 19:54:19] ax.service.ax_client: Completed trial 33 with data: {'y': (0.4898, 0.025)}.
[INFO 10-07 19:54:19] ax.service.ax_client: Generated new trial 34 with parameters {'x': 0.143255}.
[INFO 10-07 19:54:19] ax.service.ax_client: Completed trial 34 with data: {'y': (-0.088924, 0.025)}.
[INFO 10-07 19:54:19] ax.service.ax_client: Generated new trial 35 with parameters {'x': 0.053893}.
[INFO 10-07 19:54:19] ax.service.ax_client: Completed trial 35 with data: {'y': (0.045086, 0.025)}.
[INFO 10-07 19:54:19] ax.service.ax_client: Generated new trial 36 with parameters {'x': 0.231145}.
[INFO 10-07 19:54:19] ax.service.ax_client: Completed trial 36 with data: {'y': (0.000668, 0.025)}.
[INFO 10-07 19:54:20] ax.service.ax_client: Generated new trial 37 with parameters {'x': 0.314293}.
[INFO 10-07 19:54:20] ax.service.ax_client: Completed trial 37 with data: {'y': (0.028526, 0.025)}.
[INFO 10-07 19:54:20] ax.service.ax_client: Generated new trial 38 with parameters {'x': 0.904513}.
[INFO 10-07 19:54:20] ax.service.ax_client: Completed trial 38 with data: {'y': (0.758632, 0.025)}.
[INFO 10-07 19:54:20] ax.service.ax_client: Generated new trial 39 with parameters {'x': 0.903323}.
[INFO 10-07 19:54:20] ax.service.ax_client: Completed trial 39 with data: {'y': (0.737359, 0.025)}.
[INFO 10-07 19:54:20] ax.service.ax_client: Generated new trial 40 with parameters {'x': 0.905989}.
[INFO 10-07 19:54:20] ax.service.ax_client: Completed trial 40 with data: {'y': (0.746064, 0.025)}.
[INFO 10-07 19:54:21] ax.service.ax_client: Generated new trial 41 with parameters {'x': 0.905398}.
[INFO 10-07 19:54:21] ax.service.ax_client: Completed trial 41 with data: {'y': (0.875281, 0.025)}.
[INFO 10-07 19:54:21] ax.service.ax_client: Generated new trial 42 with parameters {'x': 0.899785}.
[INFO 10-07 19:54:21] ax.service.ax_client: Completed trial 42 with data: {'y': (0.817782, 0.025)}.
[INFO 10-07 19:54:21] ax.service.ax_client: Generated new trial 43 with parameters {'x': 0.90261}.
[INFO 10-07 19:54:21] ax.service.ax_client: Completed trial 43 with data: {'y': (0.845981, 0.025)}.
[INFO 10-07 19:54:21] ax.service.ax_client: Generated new trial 44 with parameters {'x': 0.904756}.
[INFO 10-07 19:54:21] ax.service.ax_client: Completed trial 44 with data: {'y': (0.803342, 0.025)}.
[INFO 10-07 19:54:22] ax.service.ax_client: Generated new trial 45 with parameters {'x': 0.903744}.
[INFO 10-07 19:54:22] ax.service.ax_client: Completed trial 45 with data: {'y': (0.84641, 0.025)}.
[INFO 10-07 19:54:22] ax.service.ax_client: Generated new trial 46 with parameters {'x': 0.904268}.
[INFO 10-07 19:54:22] ax.service.ax_client: Completed trial 46 with data: {'y': (0.778349, 0.025)}.
[INFO 10-07 19:54:22] ax.service.ax_client: Generated new trial 47 with parameters {'x': 0.901881}.
[INFO 10-07 19:54:22] ax.service.ax_client: Completed trial 47 with data: {'y': (0.825683, 0.025)}.
[INFO 10-07 19:54:23] ax.service.ax_client: Generated new trial 48 with parameters {'x': 0.905184}.
[INFO 10-07 19:54:23] ax.service.ax_client: Completed trial 48 with data: {'y': (0.804793, 0.025)}.
[INFO 10-07 19:54:23] ax.service.ax_client: Generated new trial 49 with parameters {'x': 0.903503}.
[INFO 10-07 19:54:23] ax.service.ax_client: Completed trial 49 with data: {'y': (0.791156, 0.025)}.
[INFO 10-07 19:54:23] ax.service.ax_client: Generated new trial 50 with parameters {'x': 0.903028}.
[INFO 10-07 19:54:23] ax.service.ax_client: Completed trial 50 with data: {'y': (0.885919, 0.025)}.
[INFO 10-07 19:54:23] ax.service.ax_client: Generated new trial 51 with parameters {'x': 0.90219}.
[INFO 10-07 19:54:23] ax.service.ax_client: Completed trial 51 with data: {'y': (0.824475, 0.025)}.
[INFO 10-07 19:54:23] ax.service.ax_client: Generated new trial 52 with parameters {'x': 0.901206}.
[INFO 10-07 19:54:23] ax.service.ax_client: Completed trial 52 with data: {'y': (0.781975, 0.025)}.
[INFO 10-07 19:54:24] ax.service.ax_client: Generated new trial 53 with parameters {'x': 0.905869}.
[INFO 10-07 19:54:24] ax.service.ax_client: Completed trial 53 with data: {'y': (0.812709, 0.025)}.
[INFO 10-07 19:54:24] ax.service.ax_client: Generated new trial 54 with parameters {'x': 0.906562}.
[INFO 10-07 19:54:24] ax.service.ax_client: Completed trial 54 with data: {'y': (0.774908, 0.025)}.
[INFO 10-07 19:54:24] ax.service.ax_client: Generated new trial 55 with parameters {'x': 0.904895}.
[INFO 10-07 19:54:24] ax.service.ax_client: Completed trial 55 with data: {'y': (0.822912, 0.025)}.
[INFO 10-07 19:54:24] ax.service.ax_client: Generated new trial 56 with parameters {'x': 0.904628}.
[INFO 10-07 19:54:24] ax.service.ax_client: Completed trial 56 with data: {'y': (0.753792, 0.025)}.
[INFO 10-07 19:54:25] ax.service.ax_client: Generated new trial 57 with parameters {'x': 0.904513}.
[INFO 10-07 19:54:25] ax.service.ax_client: Completed trial 57 with data: {'y': (0.781121, 0.025)}.
[INFO 10-07 19:54:25] ax.service.ax_client: Generated new trial 58 with parameters {'x': 0.905564}.
[INFO 10-07 19:54:25] ax.service.ax_client: Completed trial 58 with data: {'y': (0.865246, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:25] ax.service.ax_client: Generated new trial 59 with parameters {'x': 0.903846}.
[INFO 10-07 19:54:25] ax.service.ax_client: Completed trial 59 with data: {'y': (0.770465, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:26] ax.service.ax_client: Generated new trial 60 with parameters {'x': 0.904223}.
[INFO 10-07 19:54:26] ax.service.ax_client: Completed trial 60 with data: {'y': (0.894523, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:26] ax.service.ax_client: Generated new trial 61 with parameters {'x': 0.904085}.
[INFO 10-07 19:54:26] ax.service.ax_client: Completed trial 61 with data: {'y': (0.839171, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:26] ax.service.ax_client: Generated new trial 62 with parameters {'x': 0.901299}.
[INFO 10-07 19:54:26] ax.service.ax_client: Completed trial 62 with data: {'y': (0.861088, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:27] ax.service.ax_client: Generated new trial 63 with parameters {'x': 0.903516}.
[INFO 10-07 19:54:27] ax.service.ax_client: Completed trial 63 with data: {'y': (0.871806, 0.025)}.
[INFO 10-07 19:54:27] ax.service.ax_client: Generated new trial 64 with parameters {'x': 0.902835}.
[INFO 10-07 19:54:27] ax.service.ax_client: Completed trial 64 with data: {'y': (0.777549, 0.025)}.
[INFO 10-07 19:54:27] ax.service.ax_client: Generated new trial 65 with parameters {'x': 0.904272}.
[INFO 10-07 19:54:27] ax.service.ax_client: Completed trial 65 with data: {'y': (0.848523, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:28] ax.service.ax_client: Generated new trial 66 with parameters {'x': 0.905448}.
[INFO 10-07 19:54:28] ax.service.ax_client: Completed trial 66 with data: {'y': (0.867853, 0.025)}.
[INFO 10-07 19:54:28] ax.service.ax_client: Generated new trial 67 with parameters {'x': 0.905455}.
[INFO 10-07 19:54:28] ax.service.ax_client: Completed trial 67 with data: {'y': (0.794843, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:28] ax.service.ax_client: Generated new trial 68 with parameters {'x': 0.907614}.
[INFO 10-07 19:54:28] ax.service.ax_client: Completed trial 68 with data: {'y': (0.840878, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:29] ax.service.ax_client: Generated new trial 69 with parameters {'x': 0.904089}.
[INFO 10-07 19:54:29] ax.service.ax_client: Completed trial 69 with data: {'y': (0.702442, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:30] ax.service.ax_client: Generated new trial 70 with parameters {'x': 0.906178}.
[INFO 10-07 19:54:30] ax.service.ax_client: Completed trial 70 with data: {'y': (0.810401, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:30] ax.service.ax_client: Generated new trial 71 with parameters {'x': 0.903106}.
[INFO 10-07 19:54:30] ax.service.ax_client: Completed trial 71 with data: {'y': (0.81752, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:31] ax.service.ax_client: Generated new trial 72 with parameters {'x': 0.905011}.
[INFO 10-07 19:54:31] ax.service.ax_client: Completed trial 72 with data: {'y': (0.766349, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:31] ax.service.ax_client: Generated new trial 73 with parameters {'x': 0.908861}.
[INFO 10-07 19:54:31] ax.service.ax_client: Completed trial 73 with data: {'y': (0.775356, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:31] ax.service.ax_client: Generated new trial 74 with parameters {'x': 0.902759}.
[INFO 10-07 19:54:31] ax.service.ax_client: Completed trial 74 with data: {'y': (0.837292, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:32] ax.service.ax_client: Generated new trial 75 with parameters {'x': 0.906003}.
[INFO 10-07 19:54:32] ax.service.ax_client: Completed trial 75 with data: {'y': (0.743123, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:32] ax.service.ax_client: Generated new trial 76 with parameters {'x': 0.903692}.
[INFO 10-07 19:54:32] ax.service.ax_client: Completed trial 76 with data: {'y': (0.8025, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:33] ax.service.ax_client: Generated new trial 77 with parameters {'x': 0.902442}.
[INFO 10-07 19:54:33] ax.service.ax_client: Completed trial 77 with data: {'y': (0.765959, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:33] ax.service.ax_client: Generated new trial 78 with parameters {'x': 0.903342}.
[INFO 10-07 19:54:33] ax.service.ax_client: Completed trial 78 with data: {'y': (0.865114, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:34] ax.service.ax_client: Generated new trial 79 with parameters {'x': 0.901865}.
[INFO 10-07 19:54:34] ax.service.ax_client: Completed trial 79 with data: {'y': (0.88635, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:34] ax.service.ax_client: Generated new trial 80 with parameters {'x': 0.900483}.
[INFO 10-07 19:54:34] ax.service.ax_client: Completed trial 80 with data: {'y': (0.788329, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:35] ax.service.ax_client: Generated new trial 81 with parameters {'x': 0.903098}.
[INFO 10-07 19:54:35] ax.service.ax_client: Completed trial 81 with data: {'y': (0.774204, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:36] ax.service.ax_client: Generated new trial 82 with parameters {'x': 0.904011}.
[INFO 10-07 19:54:36] ax.service.ax_client: Completed trial 82 with data: {'y': (0.708384, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:36] ax.service.ax_client: Generated new trial 83 with parameters {'x': 0.903559}.
[INFO 10-07 19:54:36] ax.service.ax_client: Completed trial 83 with data: {'y': (0.915097, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:37] ax.service.ax_client: Generated new trial 84 with parameters {'x': 0.902283}.
[INFO 10-07 19:54:37] ax.service.ax_client: Completed trial 84 with data: {'y': (0.754071, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:37] ax.service.ax_client: Generated new trial 85 with parameters {'x': 0.900829}.
[INFO 10-07 19:54:37] ax.service.ax_client: Completed trial 85 with data: {'y': (0.797216, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:38] ax.service.ax_client: Generated new trial 86 with parameters {'x': 0.903825}.
[INFO 10-07 19:54:38] ax.service.ax_client: Completed trial 86 with data: {'y': (0.804757, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:38] ax.service.ax_client: Generated new trial 87 with parameters {'x': 0.902942}.
[INFO 10-07 19:54:38] ax.service.ax_client: Completed trial 87 with data: {'y': (0.655479, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:39] ax.service.ax_client: Generated new trial 88 with parameters {'x': 0.905061}.
[INFO 10-07 19:54:39] ax.service.ax_client: Completed trial 88 with data: {'y': (0.782868, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:40] ax.service.ax_client: Generated new trial 89 with parameters {'x': 0.905226}.
[INFO 10-07 19:54:40] ax.service.ax_client: Completed trial 89 with data: {'y': (0.828164, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:40] ax.service.ax_client: Generated new trial 90 with parameters {'x': 0.904226}.
[INFO 10-07 19:54:40] ax.service.ax_client: Completed trial 90 with data: {'y': (0.807066, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:41] ax.service.ax_client: Generated new trial 91 with parameters {'x': 0.904388}.
[INFO 10-07 19:54:41] ax.service.ax_client: Completed trial 91 with data: {'y': (0.849037, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:42] ax.service.ax_client: Generated new trial 92 with parameters {'x': 0.904888}.
[INFO 10-07 19:54:42] ax.service.ax_client: Completed trial 92 with data: {'y': (0.778917, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:42] ax.service.ax_client: Generated new trial 93 with parameters {'x': 0.905674}.
[INFO 10-07 19:54:42] ax.service.ax_client: Completed trial 93 with data: {'y': (0.811084, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:43] ax.service.ax_client: Generated new trial 94 with parameters {'x': 0.904558}.
[INFO 10-07 19:54:43] ax.service.ax_client: Completed trial 94 with data: {'y': (0.751561, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:44] ax.service.ax_client: Generated new trial 95 with parameters {'x': 0.903253}.
[INFO 10-07 19:54:44] ax.service.ax_client: Completed trial 95 with data: {'y': (0.793051, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:45] ax.service.ax_client: Generated new trial 96 with parameters {'x': 0.901473}.
[INFO 10-07 19:54:45] ax.service.ax_client: Completed trial 96 with data: {'y': (0.812853, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:46] ax.service.ax_client: Generated new trial 97 with parameters {'x': 0.904657}.
[INFO 10-07 19:54:46] ax.service.ax_client: Completed trial 97 with data: {'y': (0.862285, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:47] ax.service.ax_client: Generated new trial 98 with parameters {'x': 0.905573}.
[INFO 10-07 19:54:47] ax.service.ax_client: Completed trial 98 with data: {'y': (0.869719, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:54:47] ax.service.ax_client: Generated new trial 99 with parameters {'x': 0.90647}.
[INFO 10-07 19:54:47] ax.service.ax_client: Completed trial 99 with data: {'y': (0.820214, 0.025)}.
[INFO 10-07 19:54:47] ax.service.ax_client: Starting optimization with verbose logging. To disable logging, set the `verbose_logging` argument to `False`. Note that float values in the logs are rounded to 6 decimal points.
[INFO 10-07 19:54:47] ax.service.utils.instantiation: Inferred value type of ParameterType.FLOAT for parameter x. If that is not the expected value type, you can explicity specify 'value_type' ('int', 'float', 'bool' or 'str') in parameter dict.
[INFO 10-07 19:54:47] ax.service.utils.instantiation: Created search space: SearchSpace(parameters=[RangeParameter(name='x', parameter_type=FLOAT, range=[0.0, 1.0])], parameter_constraints=[]).
[INFO 10-07 19:54:47] ax.modelbridge.dispatch_utils: Using Bayesian optimization since there are more ordered parameters than there are categories for the unordered categorical parameters.
[INFO 10-07 19:54:47] ax.modelbridge.dispatch_utils: Using Bayesian Optimization generation strategy: GenerationStrategy(name='Sobol+GPEI', steps=[Sobol for 5 trials, GPEI for subsequent trials]). Iterations after 5 will take longer to generate due to model-fitting.
[INFO 10-07 19:54:47] ax.service.ax_client: Generated new trial 0 with parameters {'x': 0.288042}.
[INFO 10-07 19:54:47] ax.service.ax_client: Completed trial 0 with data: {'y': (0.091562, 0.025)}.
[INFO 10-07 19:54:47] ax.service.ax_client: Generated new trial 1 with parameters {'x': 0.884069}.
[INFO 10-07 19:54:47] ax.service.ax_client: Completed trial 1 with data: {'y': (0.596971, 0.025)}.
[INFO 10-07 19:54:47] ax.service.ax_client: Generated new trial 2 with parameters {'x': 0.654815}.
[INFO 10-07 19:54:47] ax.service.ax_client: Completed trial 2 with data: {'y': (-0.044755, 0.025)}.
[INFO 10-07 19:54:47] ax.service.ax_client: Generated new trial 3 with parameters {'x': 0.724609}.
[INFO 10-07 19:54:47] ax.service.ax_client: Completed trial 3 with data: {'y': (0.281227, 0.025)}.
[INFO 10-07 19:54:47] ax.service.ax_client: Generated new trial 4 with parameters {'x': 0.577848}.
[INFO 10-07 19:54:47] ax.service.ax_client: Completed trial 4 with data: {'y': (-0.015512, 0.025)}.
[INFO 10-07 19:54:48] ax.service.ax_client: Generated new trial 5 with parameters {'x': 1.0}.
[INFO 10-07 19:54:48] ax.service.ax_client: Completed trial 5 with data: {'y': (-0.057446, 0.025)}.
[INFO 10-07 19:54:48] ax.service.ax_client: Generated new trial 6 with parameters {'x': 0.834275}.
[INFO 10-07 19:54:48] ax.service.ax_client: Completed trial 6 with data: {'y': (-0.04235, 0.025)}.
[INFO 10-07 19:54:48] ax.service.ax_client: Generated new trial 7 with parameters {'x': 0.901671}.
[INFO 10-07 19:54:48] ax.service.ax_client: Completed trial 7 with data: {'y': (0.744515, 0.025)}.
[INFO 10-07 19:54:49] ax.service.ax_client: Generated new trial 8 with parameters {'x': 0.917996}.
[INFO 10-07 19:54:49] ax.service.ax_client: Completed trial 8 with data: {'y': (0.600449, 0.025)}.
[INFO 10-07 19:54:49] ax.service.ax_client: Generated new trial 9 with parameters {'x': 0.0}.
[INFO 10-07 19:54:49] ax.service.ax_client: Completed trial 9 with data: {'y': (-0.07579, 0.025)}.
[INFO 10-07 19:54:50] ax.service.ax_client: Generated new trial 10 with parameters {'x': 0.414479}.
[INFO 10-07 19:54:50] ax.service.ax_client: Completed trial 10 with data: {'y': (0.013069, 0.025)}.
[INFO 10-07 19:54:50] ax.service.ax_client: Generated new trial 11 with parameters {'x': 0.898424}.
[INFO 10-07 19:54:50] ax.service.ax_client: Completed trial 11 with data: {'y': (0.798921, 0.025)}.
[INFO 10-07 19:54:50] ax.service.ax_client: Generated new trial 12 with parameters {'x': 0.173048}.
[INFO 10-07 19:54:50] ax.service.ax_client: Completed trial 12 with data: {'y': (-0.054558, 0.025)}.
[INFO 10-07 19:54:51] ax.service.ax_client: Generated new trial 13 with parameters {'x': 0.495173}.
[INFO 10-07 19:54:51] ax.service.ax_client: Completed trial 13 with data: {'y': (0.182608, 0.025)}.
[INFO 10-07 19:54:51] ax.service.ax_client: Generated new trial 14 with parameters {'x': 0.087444}.
[INFO 10-07 19:54:51] ax.service.ax_client: Completed trial 14 with data: {'y': (-0.003875, 0.025)}.
[INFO 10-07 19:54:51] ax.service.ax_client: Generated new trial 15 with parameters {'x': 0.896601}.
[INFO 10-07 19:54:51] ax.service.ax_client: Completed trial 15 with data: {'y': (0.775562, 0.025)}.
[INFO 10-07 19:54:51] ax.service.ax_client: Generated new trial 16 with parameters {'x': 0.76859}.
[INFO 10-07 19:54:51] ax.service.ax_client: Completed trial 16 with data: {'y': (-0.019609, 0.025)}.
[INFO 10-07 19:54:51] ax.service.ax_client: Generated new trial 17 with parameters {'x': 0.900019}.
[INFO 10-07 19:54:51] ax.service.ax_client: Completed trial 17 with data: {'y': (0.800895, 0.025)}.
[INFO 10-07 19:54:52] ax.service.ax_client: Generated new trial 18 with parameters {'x': 0.903539}.
[INFO 10-07 19:54:52] ax.service.ax_client: Completed trial 18 with data: {'y': (0.742944, 0.025)}.
[INFO 10-07 19:54:52] ax.service.ax_client: Generated new trial 19 with parameters {'x': 0.897554}.
[INFO 10-07 19:54:52] ax.service.ax_client: Completed trial 19 with data: {'y': (0.902288, 0.025)}.
[INFO 10-07 19:54:52] ax.service.ax_client: Generated new trial 20 with parameters {'x': 0.895049}.
[INFO 10-07 19:54:52] ax.service.ax_client: Completed trial 20 with data: {'y': (0.851369, 0.025)}.
[INFO 10-07 19:54:53] ax.service.ax_client: Generated new trial 21 with parameters {'x': 0.941751}.
[INFO 10-07 19:54:53] ax.service.ax_client: Completed trial 21 with data: {'y': (0.297606, 0.025)}.
[INFO 10-07 19:54:53] ax.service.ax_client: Generated new trial 22 with parameters {'x': 0.894068}.
[INFO 10-07 19:54:53] ax.service.ax_client: Completed trial 22 with data: {'y': (0.808756, 0.025)}.
[INFO 10-07 19:54:53] ax.service.ax_client: Generated new trial 23 with parameters {'x': 0.345703}.
[INFO 10-07 19:54:53] ax.service.ax_client: Completed trial 23 with data: {'y': (0.062142, 0.025)}.
[INFO 10-07 19:54:53] ax.service.ax_client: Generated new trial 24 with parameters {'x': 0.895878}.
[INFO 10-07 19:54:53] ax.service.ax_client: Completed trial 24 with data: {'y': (0.898467, 0.025)}.
[INFO 10-07 19:54:53] ax.service.ax_client: Generated new trial 25 with parameters {'x': 0.895491}.
[INFO 10-07 19:54:53] ax.service.ax_client: Completed trial 25 with data: {'y': (0.834842, 0.025)}.
[INFO 10-07 19:54:54] ax.service.ax_client: Generated new trial 26 with parameters {'x': 0.894661}.
[INFO 10-07 19:54:54] ax.service.ax_client: Completed trial 26 with data: {'y': (0.801361, 0.025)}.
[INFO 10-07 19:54:54] ax.service.ax_client: Generated new trial 27 with parameters {'x': 0.897001}.
[INFO 10-07 19:54:54] ax.service.ax_client: Completed trial 27 with data: {'y': (0.811943, 0.025)}.
[INFO 10-07 19:54:54] ax.service.ax_client: Generated new trial 28 with parameters {'x': 0.89624}.
[INFO 10-07 19:54:54] ax.service.ax_client: Completed trial 28 with data: {'y': (0.765433, 0.025)}.
[INFO 10-07 19:54:54] ax.service.ax_client: Generated new trial 29 with parameters {'x': 0.893305}.
[INFO 10-07 19:54:54] ax.service.ax_client: Completed trial 29 with data: {'y': (0.671629, 0.025)}.
[INFO 10-07 19:54:54] ax.service.ax_client: Generated new trial 30 with parameters {'x': 0.866735}.
[INFO 10-07 19:54:54] ax.service.ax_client: Completed trial 30 with data: {'y': (0.328281, 0.025)}.
[INFO 10-07 19:54:55] ax.service.ax_client: Generated new trial 31 with parameters {'x': 0.698215}.
[INFO 10-07 19:54:55] ax.service.ax_client: Completed trial 31 with data: {'y': (0.444153, 0.025)}.
[INFO 10-07 19:54:55] ax.service.ax_client: Generated new trial 32 with parameters {'x': 0.237561}.
[INFO 10-07 19:54:55] ax.service.ax_client: Completed trial 32 with data: {'y': (0.054493, 0.025)}.
[INFO 10-07 19:54:55] ax.service.ax_client: Generated new trial 33 with parameters {'x': 0.897902}.
[INFO 10-07 19:54:55] ax.service.ax_client: Completed trial 33 with data: {'y': (0.874405, 0.025)}.
[INFO 10-07 19:54:55] ax.service.ax_client: Generated new trial 34 with parameters {'x': 0.683011}.
[INFO 10-07 19:54:55] ax.service.ax_client: Completed trial 34 with data: {'y': (0.428424, 0.025)}.
[INFO 10-07 19:54:55] ax.service.ax_client: Generated new trial 35 with parameters {'x': 0.897286}.
[INFO 10-07 19:54:55] ax.service.ax_client: Completed trial 35 with data: {'y': (0.686513, 0.025)}.
[INFO 10-07 19:54:56] ax.service.ax_client: Generated new trial 36 with parameters {'x': 0.90834}.
[INFO 10-07 19:54:56] ax.service.ax_client: Completed trial 36 with data: {'y': (0.700237, 0.025)}.
[INFO 10-07 19:54:56] ax.service.ax_client: Generated new trial 37 with parameters {'x': 0.913113}.
[INFO 10-07 19:54:56] ax.service.ax_client: Completed trial 37 with data: {'y': (0.794405, 0.025)}.
[INFO 10-07 19:54:57] ax.service.ax_client: Generated new trial 38 with parameters {'x': 0.961448}.
[INFO 10-07 19:54:57] ax.service.ax_client: Completed trial 38 with data: {'y': (0.03392, 0.025)}.
[INFO 10-07 19:54:57] ax.service.ax_client: Generated new trial 39 with parameters {'x': 0.524714}.
[INFO 10-07 19:54:57] ax.service.ax_client: Completed trial 39 with data: {'y': (0.102604, 0.025)}.
[INFO 10-07 19:54:57] ax.service.ax_client: Generated new trial 40 with parameters {'x': 0.710307}.
[INFO 10-07 19:54:57] ax.service.ax_client: Completed trial 40 with data: {'y': (0.442833, 0.025)}.
[INFO 10-07 19:54:57] ax.service.ax_client: Generated new trial 41 with parameters {'x': 0.464348}.
[INFO 10-07 19:54:57] ax.service.ax_client: Completed trial 41 with data: {'y': (0.056018, 0.025)}.
[INFO 10-07 19:54:58] ax.service.ax_client: Generated new trial 42 with parameters {'x': 0.045201}.
[INFO 10-07 19:54:58] ax.service.ax_client: Completed trial 42 with data: {'y': (0.023842, 0.025)}.
[INFO 10-07 19:54:58] ax.service.ax_client: Generated new trial 43 with parameters {'x': 0.129073}.
[INFO 10-07 19:54:58] ax.service.ax_client: Completed trial 43 with data: {'y': (-0.047692, 0.025)}.
[INFO 10-07 19:54:58] ax.service.ax_client: Generated new trial 44 with parameters {'x': 0.911938}.
[INFO 10-07 19:54:58] ax.service.ax_client: Completed trial 44 with data: {'y': (0.746741, 0.025)}.
[INFO 10-07 19:54:58] ax.service.ax_client: Generated new trial 45 with parameters {'x': 0.378905}.
[INFO 10-07 19:54:58] ax.service.ax_client: Completed trial 45 with data: {'y': (0.037665, 0.025)}.
[INFO 10-07 19:54:58] ax.service.ax_client: Generated new trial 46 with parameters {'x': 0.896814}.
[INFO 10-07 19:54:58] ax.service.ax_client: Completed trial 46 with data: {'y': (0.888067, 0.025)}.
[INFO 10-07 19:54:59] ax.service.ax_client: Generated new trial 47 with parameters {'x': 0.898129}.
[INFO 10-07 19:54:59] ax.service.ax_client: Completed trial 47 with data: {'y': (0.847729, 0.025)}.
[INFO 10-07 19:54:59] ax.service.ax_client: Generated new trial 48 with parameters {'x': 0.615237}.
[INFO 10-07 19:54:59] ax.service.ax_client: Completed trial 48 with data: {'y': (0.010143, 0.025)}.
[INFO 10-07 19:54:59] ax.service.ax_client: Generated new trial 49 with parameters {'x': 0.898678}.
[INFO 10-07 19:54:59] ax.service.ax_client: Completed trial 49 with data: {'y': (0.877139, 0.025)}.
[INFO 10-07 19:54:59] ax.service.ax_client: Generated new trial 50 with parameters {'x': 0.897743}.
[INFO 10-07 19:54:59] ax.service.ax_client: Completed trial 50 with data: {'y': (0.878265, 0.025)}.
[INFO 10-07 19:54:59] ax.service.ax_client: Generated new trial 51 with parameters {'x': 0.316206}.
[INFO 10-07 19:54:59] ax.service.ax_client: Completed trial 51 with data: {'y': (0.077695, 0.025)}.
[INFO 10-07 19:55:00] ax.service.ax_client: Generated new trial 52 with parameters {'x': 0.897424}.
[INFO 10-07 19:55:00] ax.service.ax_client: Completed trial 52 with data: {'y': (0.831985, 0.025)}.
[INFO 10-07 19:55:00] ax.service.ax_client: Generated new trial 53 with parameters {'x': 0.897144}.
[INFO 10-07 19:55:00] ax.service.ax_client: Completed trial 53 with data: {'y': (0.858548, 0.025)}.
[INFO 10-07 19:55:00] ax.service.ax_client: Generated new trial 54 with parameters {'x': 0.89926}.
[INFO 10-07 19:55:00] ax.service.ax_client: Completed trial 54 with data: {'y': (0.819826, 0.025)}.
[INFO 10-07 19:55:00] ax.service.ax_client: Generated new trial 55 with parameters {'x': 0.207539}.
[INFO 10-07 19:55:00] ax.service.ax_client: Completed trial 55 with data: {'y': (-0.05774, 0.025)}.
[INFO 10-07 19:55:01] ax.service.ax_client: Generated new trial 56 with parameters {'x': 0.898001}.
[INFO 10-07 19:55:01] ax.service.ax_client: Completed trial 56 with data: {'y': (0.712392, 0.025)}.
[INFO 10-07 19:55:01] ax.service.ax_client: Generated new trial 57 with parameters {'x': 0.896391}.
[INFO 10-07 19:55:01] ax.service.ax_client: Completed trial 57 with data: {'y': (0.791963, 0.025)}.
[INFO 10-07 19:55:01] ax.service.ax_client: Generated new trial 58 with parameters {'x': 0.897647}.
[INFO 10-07 19:55:01] ax.service.ax_client: Completed trial 58 with data: {'y': (0.860668, 0.025)}.
[INFO 10-07 19:55:01] ax.service.ax_client: Generated new trial 59 with parameters {'x': 0.800906}.
[INFO 10-07 19:55:01] ax.service.ax_client: Completed trial 59 with data: {'y': (-0.048659, 0.025)}.
[INFO 10-07 19:55:02] ax.service.ax_client: Generated new trial 60 with parameters {'x': 0.89893}.
[INFO 10-07 19:55:02] ax.service.ax_client: Completed trial 60 with data: {'y': (0.78515, 0.025)}.
[INFO 10-07 19:55:02] ax.service.ax_client: Generated new trial 61 with parameters {'x': 0.896707}.
[INFO 10-07 19:55:02] ax.service.ax_client: Completed trial 61 with data: {'y': (0.834243, 0.025)}.
[INFO 10-07 19:55:02] ax.service.ax_client: Generated new trial 62 with parameters {'x': 0.263255}.
[INFO 10-07 19:55:02] ax.service.ax_client: Completed trial 62 with data: {'y': (0.046619, 0.025)}.
[INFO 10-07 19:55:02] ax.service.ax_client: Generated new trial 63 with parameters {'x': 0.897215}.
[INFO 10-07 19:55:02] ax.service.ax_client: Completed trial 63 with data: {'y': (0.742342, 0.025)}.
[INFO 10-07 19:55:03] ax.service.ax_client: Generated new trial 64 with parameters {'x': 0.549838}.
[INFO 10-07 19:55:03] ax.service.ax_client: Completed trial 64 with data: {'y': (-0.038009, 0.025)}.
[INFO 10-07 19:55:03] ax.service.ax_client: Generated new trial 65 with parameters {'x': 0.897077}.
[INFO 10-07 19:55:03] ax.service.ax_client: Completed trial 65 with data: {'y': (0.780135, 0.025)}.
[INFO 10-07 19:55:03] ax.service.ax_client: Generated new trial 66 with parameters {'x': 0.898229}.
[INFO 10-07 19:55:03] ax.service.ax_client: Completed trial 66 with data: {'y': (0.807194, 0.025)}.
[INFO 10-07 19:55:03] ax.service.ax_client: Generated new trial 67 with parameters {'x': 0.896913}.
[INFO 10-07 19:55:03] ax.service.ax_client: Completed trial 67 with data: {'y': (0.816912, 0.025)}.
[INFO 10-07 19:55:04] ax.service.ax_client: Generated new trial 68 with parameters {'x': 0.439848}.
[INFO 10-07 19:55:04] ax.service.ax_client: Completed trial 68 with data: {'y': (0.051633, 0.025)}.
[INFO 10-07 19:55:04] ax.service.ax_client: Generated new trial 69 with parameters {'x': 0.895575}.
[INFO 10-07 19:55:04] ax.service.ax_client: Completed trial 69 with data: {'y': (0.782044, 0.025)}.
[INFO 10-07 19:55:04] ax.service.ax_client: Generated new trial 70 with parameters {'x': 0.89963}.
[INFO 10-07 19:55:04] ax.service.ax_client: Completed trial 70 with data: {'y': (0.849535, 0.025)}.
[INFO 10-07 19:55:04] ax.service.ax_client: Generated new trial 71 with parameters {'x': 0.897822}.
[INFO 10-07 19:55:04] ax.service.ax_client: Completed trial 71 with data: {'y': (0.794258, 0.025)}.
[INFO 10-07 19:55:05] ax.service.ax_client: Generated new trial 72 with parameters {'x': 0.897348}.
[INFO 10-07 19:55:05] ax.service.ax_client: Completed trial 72 with data: {'y': (0.694495, 0.025)}.
[INFO 10-07 19:55:05] ax.service.ax_client: Generated new trial 73 with parameters {'x': 0.900745}.
[INFO 10-07 19:55:05] ax.service.ax_client: Completed trial 73 with data: {'y': (0.746138, 0.025)}.
[INFO 10-07 19:55:05] ax.service.ax_client: Generated new trial 74 with parameters {'x': 0.896494}.
[INFO 10-07 19:55:05] ax.service.ax_client: Completed trial 74 with data: {'y': (0.807858, 0.025)}.
[INFO 10-07 19:55:06] ax.service.ax_client: Generated new trial 75 with parameters {'x': 0.898547}.
[INFO 10-07 19:55:06] ax.service.ax_client: Completed trial 75 with data: {'y': (0.849761, 0.025)}.
[INFO 10-07 19:55:06] ax.service.ax_client: Generated new trial 76 with parameters {'x': 0.897497}.
[INFO 10-07 19:55:06] ax.service.ax_client: Completed trial 76 with data: {'y': (0.811378, 0.025)}.
[INFO 10-07 19:55:07] ax.service.ax_client: Generated new trial 77 with parameters {'x': 0.898309}.
[INFO 10-07 19:55:07] ax.service.ax_client: Completed trial 77 with data: {'y': (0.853147, 0.025)}.
[INFO 10-07 19:55:07] ax.service.ax_client: Generated new trial 78 with parameters {'x': 0.894768}.
[INFO 10-07 19:55:07] ax.service.ax_client: Completed trial 78 with data: {'y': (0.867256, 0.025)}.
[INFO 10-07 19:55:08] ax.service.ax_client: Generated new trial 79 with parameters {'x': 0.914538}.
[INFO 10-07 19:55:08] ax.service.ax_client: Completed trial 79 with data: {'y': (0.705969, 0.025)}.
[INFO 10-07 19:55:08] ax.service.ax_client: Generated new trial 80 with parameters {'x': 0.878624}.
[INFO 10-07 19:55:08] ax.service.ax_client: Completed trial 80 with data: {'y': (0.574359, 0.025)}.
[INFO 10-07 19:55:09] ax.service.ax_client: Generated new trial 81 with parameters {'x': 0.895336}.
[INFO 10-07 19:55:09] ax.service.ax_client: Completed trial 81 with data: {'y': (0.803395, 0.025)}.
[INFO 10-07 19:55:09] ax.service.ax_client: Generated new trial 82 with parameters {'x': 0.89535}.
[INFO 10-07 19:55:09] ax.service.ax_client: Completed trial 82 with data: {'y': (0.867458, 0.025)}.
[INFO 10-07 19:55:10] ax.service.ax_client: Generated new trial 83 with parameters {'x': 0.690545}.
[INFO 10-07 19:55:10] ax.service.ax_client: Completed trial 83 with data: {'y': (0.438165, 0.025)}.
[INFO 10-07 19:55:10] ax.service.ax_client: Generated new trial 84 with parameters {'x': 0.896}.
[INFO 10-07 19:55:10] ax.service.ax_client: Completed trial 84 with data: {'y': (0.674365, 0.025)}.
[INFO 10-07 19:55:11] ax.service.ax_client: Generated new trial 85 with parameters {'x': 0.900753}.
[INFO 10-07 19:55:11] ax.service.ax_client: Completed trial 85 with data: {'y': (0.763926, 0.025)}.
[INFO 10-07 19:55:11] ax.service.ax_client: Generated new trial 86 with parameters {'x': 0.92605}.
[INFO 10-07 19:55:11] ax.service.ax_client: Completed trial 86 with data: {'y': (0.586525, 0.025)}.
[INFO 10-07 19:55:12] ax.service.ax_client: Generated new trial 87 with parameters {'x': 0.900404}.
[INFO 10-07 19:55:12] ax.service.ax_client: Completed trial 87 with data: {'y': (0.821369, 0.025)}.
[INFO 10-07 19:55:12] ax.service.ax_client: Generated new trial 88 with parameters {'x': 0.897417}.
[INFO 10-07 19:55:12] ax.service.ax_client: Completed trial 88 with data: {'y': (0.929901, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:55:13] ax.service.ax_client: Generated new trial 89 with parameters {'x': 0.899462}.
[INFO 10-07 19:55:13] ax.service.ax_client: Completed trial 89 with data: {'y': (0.825679, 0.025)}.
[INFO 10-07 19:55:13] ax.service.ax_client: Generated new trial 90 with parameters {'x': 0.897562}.
[INFO 10-07 19:55:13] ax.service.ax_client: Completed trial 90 with data: {'y': (0.807654, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:55:14] ax.service.ax_client: Generated new trial 91 with parameters {'x': 0.89882}.
[INFO 10-07 19:55:14] ax.service.ax_client: Completed trial 91 with data: {'y': (0.807801, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:55:15] ax.service.ax_client: Generated new trial 92 with parameters {'x': 0.898812}.
[INFO 10-07 19:55:15] ax.service.ax_client: Completed trial 92 with data: {'y': (0.815527, 0.025)}.
/opt/conda/lib/python3.9/site-packages/botorch/optim/optimize.py:306: RuntimeWarning:
Optimization failed in `gen_candidates_scipy` with the following warning(s):
[NumericalWarning('A not p.d., added jitter of 1.0e-08 to the diagonal'), OptimizationWarning('Optimization failed within `scipy.optimize.minimize` with status 2.'), NumericalWarning('A not p.d., added jitter of 1.0e-08 to the diagonal')]
Trying again with a new set of initial conditions.
[INFO 10-07 19:55:16] ax.service.ax_client: Generated new trial 93 with parameters {'x': 0.897152}.
[INFO 10-07 19:55:16] ax.service.ax_client: Completed trial 93 with data: {'y': (0.76805, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:55:16] ax.service.ax_client: Generated new trial 94 with parameters {'x': 0.899105}.
[INFO 10-07 19:55:16] ax.service.ax_client: Completed trial 94 with data: {'y': (0.755474, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:55:17] ax.service.ax_client: Generated new trial 95 with parameters {'x': 0.897739}.
[INFO 10-07 19:55:17] ax.service.ax_client: Completed trial 95 with data: {'y': (0.816547, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:55:18] ax.service.ax_client: Generated new trial 96 with parameters {'x': 0.896055}.
[INFO 10-07 19:55:18] ax.service.ax_client: Completed trial 96 with data: {'y': (0.7626, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:55:18] ax.service.ax_client: Generated new trial 97 with parameters {'x': 0.899258}.
[INFO 10-07 19:55:18] ax.service.ax_client: Completed trial 97 with data: {'y': (0.735215, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:55:19] ax.service.ax_client: Generated new trial 98 with parameters {'x': 0.898537}.
[INFO 10-07 19:55:19] ax.service.ax_client: Completed trial 98 with data: {'y': (0.74876, 0.025)}.
/opt/conda/lib/python3.9/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:
A not p.d., added jitter of 1.0e-08 to the diagonal
[INFO 10-07 19:55:20] ax.service.ax_client: Generated new trial 99 with parameters {'x': 0.899035}.
[INFO 10-07 19:55:20] ax.service.ax_client: Completed trial 99 with data: {'y': (0.848104, 0.025)}.
# Plot with error bars!
y_trials_stacked = np.maximum.accumulate(np.array(y_trials).T)
import plotly.graph_objects as go
fig = go.Figure()
fig.add_scatter(
x=df_random["num_samples"],
y=df_random["objective"],
error_y={"array": df_random["stdev"]},
name="Random sampling",
)
fig.add_scatter(
x=df_latinhypercube["num_samples"],
y=df_latinhypercube["objective"],
error_y={"array": df_latinhypercube["stdev"]},
name="Latin Hypercube Sampling",
)
fig.add_scatter(
x=df_sobol["num_samples"],
y=df_sobol["objective"],
error_y={"array": df_sobol["stdev"]},
name="Sobol sampling",
)
fig.add_scatter(
x=list(range(len(y_trials_stacked))),
y=y_trials_stacked[:, 0],
name="Ax Sobol+GPEI Run 1",
)
fig.add_scatter(
x=list(range(len(y_trials_stacked))),
y=y_trials_stacked[:, 1],
name="Ax Sobol+GPEI Run 2",
)
fig.add_scatter(
x=list(range(len(y_trials_stacked))),
y=y_trials_stacked[:, 2],
name="Ax Sobol+GPEI Run 3",
)
fig.add_scatter(
x=list(range(len(y_trials_stacked))),
y=y_trials_stacked[:, 3],
name="Ax Sobol+GPEI Run 4",
)
fig.add_scatter(
x=list(range(len(y_trials_stacked))),
y=y_trials_stacked[:, 4],
name="Ax Sobol+GPEI Run 5",
)
!pip install optuna
Collecting optuna
Downloading optuna-3.0.2-py3-none-any.whl (348 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 348.3/348.3 kB 5.4 MB/s eta 0:00:0000:01
?25hRequirement already satisfied: alembic>=1.5.0 in /opt/conda/lib/python3.9/site-packages (from optuna) (1.8.1)
Requirement already satisfied: sqlalchemy>=1.3.0 in /opt/conda/lib/python3.9/site-packages (from optuna) (1.4.41)
Requirement already satisfied: numpy in /opt/conda/lib/python3.9/site-packages (from optuna) (1.23.3)
Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.9/site-packages (from optuna) (21.3)
Collecting cliff
Downloading cliff-4.0.0-py3-none-any.whl (80 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 81.0/81.0 kB 5.4 MB/s eta 0:00:00
?25hCollecting colorlog
Downloading colorlog-6.7.0-py2.py3-none-any.whl (11 kB)
Requirement already satisfied: tqdm in /opt/conda/lib/python3.9/site-packages (from optuna) (4.64.1)
Collecting scipy<1.9.0,>=1.7.0
Downloading scipy-1.8.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (42.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 42.2/42.2 MB 12.5 MB/s eta 0:00:0000:0100:01
?25hRequirement already satisfied: PyYAML in /opt/conda/lib/python3.9/site-packages (from optuna) (6.0)
Collecting cmaes>=0.8.2
Downloading cmaes-0.8.2-py3-none-any.whl (15 kB)
Requirement already satisfied: Mako in /opt/conda/lib/python3.9/site-packages (from alembic>=1.5.0->optuna) (1.2.3)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /opt/conda/lib/python3.9/site-packages (from packaging>=20.0->optuna) (3.0.9)
Requirement already satisfied: greenlet!=0.4.17 in /opt/conda/lib/python3.9/site-packages (from sqlalchemy>=1.3.0->optuna) (1.1.3)
Collecting PrettyTable>=0.7.2
Downloading prettytable-3.4.1-py3-none-any.whl (26 kB)
Collecting autopage>=0.4.0
Downloading autopage-0.5.1-py3-none-any.whl (29 kB)
Requirement already satisfied: importlib-metadata>=4.4 in /opt/conda/lib/python3.9/site-packages (from cliff->optuna) (4.11.4)
Collecting stevedore>=2.0.1
Downloading stevedore-4.0.0-py3-none-any.whl (49 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 49.5/49.5 kB 3.1 MB/s eta 0:00:00
?25hCollecting cmd2>=1.0.0
Downloading cmd2-2.4.2-py3-none-any.whl (147 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 147.1/147.1 kB 8.2 MB/s eta 0:00:00
?25hCollecting pyperclip>=1.6
Downloading pyperclip-1.8.2.tar.gz (20 kB)
Preparing metadata (setup.py) ... ?25ldone
?25hRequirement already satisfied: attrs>=16.3.0 in /opt/conda/lib/python3.9/site-packages (from cmd2>=1.0.0->cliff->optuna) (21.4.0)
Requirement already satisfied: wcwidth>=0.1.7 in /opt/conda/lib/python3.9/site-packages (from cmd2>=1.0.0->cliff->optuna) (0.2.5)
Requirement already satisfied: zipp>=0.5 in /opt/conda/lib/python3.9/site-packages (from importlib-metadata>=4.4->cliff->optuna) (3.8.1)
Collecting pbr!=2.1.0,>=2.0.0
Downloading pbr-5.10.0-py2.py3-none-any.whl (112 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 112.4/112.4 kB 7.0 MB/s eta 0:00:00
?25hRequirement already satisfied: MarkupSafe>=0.9.2 in /opt/conda/lib/python3.9/site-packages (from Mako->alembic>=1.5.0->optuna) (2.1.1)
Building wheels for collected packages: pyperclip
Building wheel for pyperclip (setup.py) ... ?25ldone
?25h Created wheel for pyperclip: filename=pyperclip-1.8.2-py3-none-any.whl size=11123 sha256=1690569b680429b0b7a5a9ac0404b3022ba8ee814c182e9c3add54925831eec7
Stored in directory: /home/jovyan/.cache/pip/wheels/0c/09/9e/49e21a6840ef7955b06d47394afef0058f0378c0914e48b8b8
Successfully built pyperclip
Installing collected packages: pyperclip, scipy, PrettyTable, pbr, colorlog, cmd2, cmaes, autopage, stevedore, cliff, optuna
Attempting uninstall: scipy
Found existing installation: scipy 1.9.1
Uninstalling scipy-1.9.1:
Successfully uninstalled scipy-1.9.1
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
matminer 0.7.8 requires pymongo>=4.1.1, which is not installed.
matminer 0.7.8 requires jsonschema>=4.5.1, but you have jsonschema 3.2.0 which is incompatible.
ax-platform 0.2.7.2 requires botorch==0.7.1, but you have botorch 0.7.2 which is incompatible.
Successfully installed PrettyTable-3.4.1 autopage-0.5.1 cliff-4.0.0 cmaes-0.8.2 cmd2-2.4.2 colorlog-6.7.0 optuna-3.0.2 pbr-5.10.0 pyperclip-1.8.2 scipy-1.8.1 stevedore-4.0.0